Can we stop with the (inappropriate) gatekeeping?

It’s another week, so it’s time for everyone’s favourite game: Gatekeeping.

In particular this example Chloe (a Senior Developer Advocate for Microsoft who does some cool stuff with code, while putting up with being a woman in tech on twitter) posted this:

Now there are a whole variety of reasons for this being a good thing, there’s evidence that diverse teams, while sometimes being worse at doing repetitive/samey tasks than less diverse teams, when thrown new problems do better.

Also, having people who aren’t white comp-sci males on a team leads to picking up on things, like an awareness of how your product might be mis-used. Abusers have used Venmo to send money to their victims, because “why would you want to stop someone sending you money”.

Of course, a man was here to quibble advise:

Now, machine-learning is an interesting discipline to pop up and claim that inexperienced people aren’t going to do a good job… we’ll go into that in a second.

Yes, it’s probably true that someone starting out will not be able to generate an entirely new model. But will they be able to follow tutorials and train one of the existing models? Likely yes.

Will they be able to replicate the many mistakes that ‘pro-fess-ion-al’ machine learning engineers have? Absolutely.

Machine learning has been used to codify our biases. Facial recognition performs worse on non-white faces… “flight risk assessment algorithms” which are commercially sensitive so can’t be audited, seem to report that certain communities are more of a risk.

Meanwhile there was that time that a “cancer detection” model, had actually been trained itself to detect the different colour of slide-frames that were used between control and malignant samples.

I’m just saying, that maybe Machine Learning isn’t yet the rigorous pillar of integrity and correctness that needs protection to preserve its pureness.

“React is for n00bs”

This is another good one.

When new devs start out and they use react, a variety of callouts appear:

  • “It’s too complicated, they need to learn the basics”
  • “React is too heavy, they need to learn to optimise”
  • “the amount of javascript we use on the web is too high and a security risk”
  • “if you don’t learn the basics of DOM manipulation how can you possibly do it well”
  • Server-side rendering of client-side apps is just a return to the old way
  • We shouldn’t be building apps on the web

Most of these are true to a greater or lesser extent, but you know what else is true?

This is what the web looks like now…

It is not where any of us would probably start, but it’s where we are.

Having architected a business system that uses React as the UI, that system would have been painfully unusable if every interaction was a page load on form reload… modal popups and API calls made it a better experience for users.

“They’re building unoptimised systems and that’s not good”

That is also true, however how do you learn to build an optimised system?

You ship something that gets to the point is needs to be optimised. Many systems don’t need to be… Good enough, is, well, good enough.

These things are analogous to scaling problems: if you get them, they’re nice to have.

We do want some gatekeeping

I don’t want a newbie coder to write the control software for a nuclear reactor… This is unlikely

But more realistically, the area that we need to find ways to help new programmers about about the basics of security.

I don’t want a newbie writing a user registration system, there are plenty of managed Identity Providers (IDP) out there like Auth0, Cognito, AzureAD, Login with Google, Login with Apple etc…

So yes, I wouldn’t want a newbie writing an IDP of any complexity, I can see them storing passwords in cleartext in a mysql database.

But we don’t talk about these things, or how we can give new programmers an intro to the “easy” 80% of security things: basic security on APIs, not storing secrets in your app, not using sequential/predictable IDs around the place.

It’s much more foundational “go and learn enough before we deem you WORTHY of writing for the web”.

Some people learn by doing a CompSci degree. I have one of those.

While it taught me a bunch of formal things, so much of what I’ve learned is by working with good people, making mistakes, and learning more.

I learned React in part because I was working with a bunch of coders who were learning it…As an old school HTML, JS, JQuery & CSS person, I was initially confused and scared of it. Then create-react-app appeared and I finally got it.

If we don’t turn down this obsession of gatekeeping entry, we don’t let new people learn.

We end-up with the same faces, and products will be worse for everyone. Us older-school people will get stale, stagnate and just write the same stuff until we get retired.

A feature request for LinkedIn

I like LinkedIn, but I would love if I could make recruitment messages more relevant.

I’m about to whine about recruitment, which I understand isn’t great when many good people are looking for work.

If you can do anything to help people in your network, recommendations, connecting people up – now is the time to lower your reputation-risk considerations (what if they aren’t a match, aren’t good) and do it anyway.

Although I dislike the Storification of LinkedIn, and find “Heart Warming Stories of Dubious Origin, About That One Time Someone Showed Basic Human Empathy” posts a little grating, I like LinkedIn.

I primarily work in the Media & Entertainment industry, and very often people move around. One time I was working with a team who were re-engineering a high profile transcode stack, and we needed to check compatibility that one consumer with very Fussy Set-Top Boxes specific H264 encoding parameters.

Searching on LinkedIn found that someone I’d previously worked with was now there, and that was one of those useful back-channels that actually get the work done, alongside the formal ones where invariably detail is lost in all the mediation layers.

I’ve previously found work through LinkedIn also, people in my network were looking and we had chats…

In both of these cases it was a route to contact people who I likely wouldn’t have managed otherwise.

The Bit Where I Bitch About Recruiters

While I know #NotAllRecruiters, many are somewhat annoying.

I’m quite specific in my profile intro of the kind of roles I’m open to, and still I get requests to be a: Permanent, SAP, Project Manager, in Bracknell.

That’s one technology I’ve never worked with (merely around) and 3 job qualities that I will avoid.

Tiresome for everyone, a waste of my time to read and theirs to send.

The over-engineered solution

As mentioned, I’ve a number of relatively simple conditions about jobs I’ll consider.

One time I got a message about a job that was “Only for Oxbridge graduates, but Imperial is also OK” – I know this was meant to be flattering and give the impression of an intellectual workplace (while also being a bit negging that “Imperial was almost good enough”). However, it just screamed of a horrendously toxic culture with Platinum Grade Gatekeeping.

So if you’re specific about what you’re looking for why don’t you get to state that in some questions, and when a recruiter who isn’t in your network wants to contact you, how about they’re given a page like this… (please excuse the ๐Ÿ’ฉ mock)

A list of questions a recruiter might face: is the position permanent or contract, using appropriate technologies, what the salary is

Actually Maybe This Is Application for ML…

As I was writing this (helpfully after doing the ๐Ÿ’ฉ mockup), I thought of a much better solution: If you can choose from a smaller range of criteria – and ones that could be detected by an ML classifier – LinkedIn could just run the classifiers you care about on an “out of network” message.

The score of the message could then drive a traffic light system: the message is accepted, outright denied, and if borderline the sender needs to click a “Yes, it’s appropriate and your classier is wrong, scouts honour, promise” button.

Would it work?

Unless there was a penalty for clicking “This Isn’t Spam” I doubt it would.

I also suspect it would hurt LinkedIn’s revenue too much, if having paid for Gold Premium Ultra, people aren’t able to send messages

To the good recruiters, who like great project managers are rare but invaluable – I’m sorry.

To the rest of you, I’m just not ready to do SAP in Bracknell.

Good Luck Microsoft

Microsoft have appointed Satya Nadella as their new CEO. He’s an internal hire, but from the services bit which includes Azure.

Microsoft have appointed Satya Nadellaย as their new CEO. He’s an internal hire, but from the services bit which includes Azure. Although everybody is playing catch up to Amazon Web Services, Azure has a number of features that are interesting: getting that cloud computing isn’t just about easy access to disposable servers.

Microsoft today is like the uncle who’s wasย great when you were a kid, got you interested in stuff, and has now fallen on hard times.

Maybe I’m just biased because I like Office (which makes me a minority I know), but I don’t want a world where there isn’t Microsoft. Google Docs is great for sharing or collaboration documents, Apples iWork is great for simple documents, and well I’m sure OpenOffice is good for something.

Microsoft Research produces so many good ideas, or clever ideas, or just the plain “hey we had a random idea” ideas. They don’t many to use that many of these, so many of them are impractical with current tech. But the ideas are there, at some level the company still tries to innovate.

That innovation doesn’t come easily however, as Windows 8 and attempts for a converged desktop/mobile/tablet interface have shown. The company doesn’t have that Apple confidence of “this is the way we scroll now”. Appeasing the fans of the legacy will not help them move on. Perhaps when the company has a better idea of what the “new” Microsoft is, selling those ideas will be a bit easier.

I may well be a Mac and iOS user now, but I think if I was going to switch phones, it would be for a Windows Phone. A bit like the Palm Pre, or Blackberry’s ultimately doomed Blackberry 10 operating system, Windows Phone didn’t feel like it started off with the requirement “be like iOS”. Android and iOS are really converging in many ways, features hopping from one to the other.

For that reason alone, I would like Microsoft to do well in the future: much like the Shuttle’s fifth computer, I think we need a strong third platform in the mobile market.


Google makes VM Immortal – but how useful?

Google let you migrate machines between data-centres while they still run

While it’s a nice feature, and something that VMWare has been able to do for a while – But I can’t help feeling it’s an anti-pattern in cloud-infrastructures. Yes there are some applications that you can’t easily design as message consuming stateless data-beasts – in general to take advantage of scaling (for capacity or to money), you need to design your applications so that they can survive machine failure, be it from chaos monkey or otherwise.

Anatomy of Ticketing ‘Fail’

Having failed to get tickets for something in an annoying day of pressing reload, I try to write something constructive about scaling for big things

(Or what happens when a company that isn’t eventbrite tries to be eventbrite)

A friend wanted to book some tickets for an event. I had some time today, so I said I’d book them.

For reasons of politeness, I’m not going to name the company. The event was massively over-subscribed, there were always going to be people who were annoyed (kinda like the Olympics). I’m just annoyed because I saw things done generally quite ad-hoc, specific technical bugs hit me.

Tickets were delivered in tranches. This is a sure sign there will be massive peaks in demand…

The hour arrives, and, in your all too typical scenario: rapidly stopped responding.

A few minutes later everything went 403’d as they killed all access on the server to get the load down. Not great, but it’s a sign somebody is looking at the problem. then starts redirecting to with all the individual ticket pages iFramed through to an Amazon EC2 instance (

The IDs for the events were sequential, some had already been released, and you started to think that people had been gaming the system and ordering tickets prior to their availability windows. This was later denied by the company (which I accept), but given the way the scaling was going, at that time it was all too easy to think they were using security by obscurity to prevent access to the events.

Later in the day, when tickets appeared, it was announced via a tweet. The tweet though didn’t link to the site, but to a mailing list post, which again didn’t reference the actual site.

The site had now changed, was redirecting to again passing through to EC2 instance. Later many people complained on Facebook that they were looking the old page and pressing reload.

Anyway, I tapped. I was at the gym. I was on my iPhone. But I know my credit card number, I know my paypal password, I can even use that tiny keyboard. I’ve topped my starbucks card up while in the queue. I can do this.

There was even a mobile site.

Only the mobile site was erroring because it was asking for a non-existent field/table. I had no way to change my user-agent (and wouldn’t have trusted Opera with my credentials), and in the 10 minutes it took me to get back to my laptop all the tickets had been sold.

No tickets for me+mate. Grumbly me having seen things done badly.

As many will say, this is not life and death – but is primarly not a ticketing company, and that showed today.

If you’re going to compete with the likes of eventbrite, you’re going to have to be as good as eventbrite.

The Constructive “What can we learn” Bit

1. Believe it could happen, no matter how unbelievable.

Ask yourself “if we get off-the-scale load how will we fix it”. Working out volumetrics and scaling is hard, so alongside your “reasonable” load calculations of “we can turn off these bits of our site”, have your plans of “how you’ll move to something big, cloudy and scalable if the unbelievable happens”.

Are there components that you should move upfront? You have something like 15 to 30 minutes of goodwill. What do you need to do upfront, so that in that downtime you can come up fully scaled.

If you’re looking a scalable elastic thing, look at how much it costs to start in that state anyway.

2. Architect things to give you agility

If you can’t host all your website on a scalable platform: Subdomains, DNS expiry times and proxy-passes give you room to move, but only if set up ahead of time.

Had been available, wouldn’t have had to disappear as it has done until tomorrow. You don’t want your website down for that length of time.

DNS changes can take time, much less if you dial down the expiry times, but again you have to do prior to the event. Amazon’s route 53 is cheap so move the domains ahead of time, and set Times to Live appropriately.

While you’re waiting for that propagation, proxy-passing can be a useful technique to bounce the traffic to the new server, while the DNS propagates. Proxy passing also means that could have been redirected, rather than an entire domain.

Are you caching what you can at an HTTP level with varnish or a service level with memcache?

3. Be careful sending people onto new URLs that won’t update

Taking the ticketing system off their main website was a good move, but the static page should have remained there. The second you redirected to cloudfront, they were then looking at a page that would get stale.

Many people would have pressed reload, expecting it to appear, but they didn’t because as you can see from above above, the URL changed. They could have used the Cloudfront revocation API, but this wasn’t used.

4. Remember data protection issues

This company used the Virginia data centre (which I think is the AWS default). Without going into the whole world of pain that is data-protection and EU borders – Dublin would have less latent and less problematic compliance wise.

5. Testing is good, as is automatic deployemnt

There were not many tickets and the loading was huge, those were not avoidable. I can’t say the same about the erroring mobile site, that should not have occurred.

6. Rehearse

It’s not fun doing disaster recovery, but if you’re receiving catastrophic load then that is what you’re doing.

Write the script. Have someone else test it.

It’s not a valid plan until you have shown it works.

Transition periods are the worst: technology, privacy and injunctions

Technology is disrupting privacy in a way that we can’t fight back from, will it all be easier once we just accept it?

Transitional times are the worst. Much like the music industry trying to retain their existing business model based on recorded music, or broadcasters using DRM to maintain rights windows on content that is transmitted in-the-clear; it’s always difficult to move on. Once you’ve accepted change, it might not be as easy as it was before, but you’re at least not fighting the inevitable.

We’re currently fighting that battle with privacy. As people tag us in Facebook, other people check us into insalubrious venues, we’re stuck in an ongoing battle to remove things that we don’t want stuck to our profile. We hide behind privacy settings on sites, only to watch a friend share a private RSS feed or one poorly-written API client leaking all the information to google. Our friends re-tweet from private accounts disclosing partially-incriminating thoughts. Strangers can sometimes see one-side of a conversation, not enough to know exactly what was said, but certainly enough for my mum to admonish me for some months ago.

Today we’ve had fun with super-injunctions, Twitter and parliamentary privilege. English courts trying to uphold rulings that Scotland and the Peoples’ Republic of Twitter are not subject to. And sure the identity of CTB is a nice bit of gossipy tittle-tattle, but what about when it’s the name of someone accused of a serious crime?

Our reporting restrictions are far more extensive than those of America, and whileย I don’t want to routinely have ‘perp-walks’ in the UK, I’d rather not have trials abandoned because our protections are unworkable in the modern world.

Away from the legal sphere, with the rise of computer vision and recognition projects, (look at the flurry of activity around the Kinect), and the availability of powerful on-demand computing resources (like GPU heavy instances from Amazon), privacy will soon be a problem that can be brute-forced away. Facebook is already rolling out photo recognition (this does seem to be taking longer than most of their phased roll-outs as I know a few people who had it months ago).

Embarrassing images we thought ‘anonymous’ because the face wasn’t shown will be tied down to people through bizarre combinations of EXIF tags, 3d room mapping, carpet recognition and host of other recognition metric that I can’t even imagine. That mole on your chest will no longer just be a minor cancer risk; it’s a data point that can be correlated.

Anyway, we’re in the transitional phase: We’re still trying to hold onto old-models of privacy which in a few years won’t be possible to have without moving to the “Google Opt-Out town“.

The other side of this transition we’ll probably have less privacy, but nobody will really have privacy, and somehow that will make it alright – that or we’ll have to change our names after we leave university, and dispose of all of our electrical devices, have that mole removed, and if we want to run for political office be very careful what we get up-to at college.

dConstruct 2010 in summary

A summary of the speakers at dConstruct 2010.

I attended my first dConstruct yesterday, which was a nice trip to Brighton, seeing friends and drinking a bit too much. My very brief take on the sessions:

The Designful Company
You’re aiming for different but good, but it takes guts to aim for and will test badly as it’s unfamiliar. Sticking with the familiar will test better, but ultimately market badly.

Boil, Simmer, Reduce
Collect ideas, play with them without pre-conceptions “Nobody will die doing this”, prune stuff back again, try and aim for simplicity.

Information is Beautiful
There’s too much information, this needs to be pruned and presented. Don’t do circular diagrams.

The Power & Beauty of typography
Typefaces have emotions and these should be in keeping with the overall site/copy. Much like shoes can set off or kill an outfit. This wasn’t too well received too well by a bunch of men and women attendees who appeared to be of the “what’s wrong with 1 pair of shoes and helvetica” crowd.

The Auteur Theory of Design
Explored the idea that projects need one ultimate authority like the Film Director with Final Cut, and that person is a limiting function on the output, and can drag up or down the overall output.

Jam Session: What Improvisation Can Teach Us About Design
Improvisation can bring about your best ideas, works best within a framework when you’re riffing off other people, and your self-inhibitions are lowered when you do it; removing that self-censorship can lead to new things.

The Value of Ruins
An unexpected standout for me; archives are potentially amazing for the future, but as we turn off things like Geocities we’re potentially losing just as much information as previous civilisations did in fires and the like.

Everything The Network Touches
Eons ago a cunning road network provided the ability to carry messages really quickly, and that communication gave empires advantages. Now we’re potentially building the infrastructure for this kind of stuff in the online world, with Bathroom Scales that tweet. Every time devices get more connected information becomes increasingly contextualised and ever more useful. Winner of the Most OCD-ly amazing slide Deck Animations Award – they really were lovely.

Kerning, Orgasms & Those Goddamned Japanese Toothpicks
Nerds care about things that other people don’t. That’s fine, don’t expect them to, try and make stuff so they don’t. Never get complacent, useful feedback probably hurts. Put the “narcism of minor differences” aside to deliver

I miss being bored

In an age where you’ve always got an unread count, are you ever bored. Are we losing something because of that?

I’m a victim of CPA. I’m on Twitter. I’m Facebook. I’m still on IRC. I think that clicking “read all” in Google Reader is somehow cheating, so I’m left constantly playing Whack-a-mole on that and my iTunes podcast collection.

Anyway, the other thing I miss is genuine boredom. I can’t think when I last really said “I’m bored” and meant it, like you did when growing up. Leaving aside twee comments from teachers like “Bored people are boring”, the thing that I wonder is “Do you need to be bored sometimes”?

I don’t think this for any great spiritual reasons, I’m not going to suggest that we all go off on Vision Quests to find our Spirit Guides. I just think that sometimes, for some people, boredom is a driver to change things. In the words of some great children’s telly of the UK of the eighties: “Why don’t you turn the TV off and do something less boring instead.”

The never ending stream of messages, podcasts, feed items and conversations to tune into mean then you’re never bored. You’re sat there, sipping away at the information passing by, sating your CPA appetite but remaining ultimately unsatisfied. Sure you have close run-ins with boredom, but thanks to the omni-present inbox you can dodge it for another 30 minutes staring at Keyboard Cat on YouTube or debating constructively with like-minded individuals.

I’ll admit counter-point is that you can see lots of cool things online that you “might” want to do, but then you see so many cool things that you might want to do that choosing to actually do one of them becomes another exercise in itself (though you can always ask Twitter followers what to do)…

I think this year I need to try and unwire a little bit, feel a little more bored sometimes.