I started working at IBM on 6th August 2003. I’m feeling nostalgic as my eighteenth anniversary approaches, so wanted to write about what I’ve been doing all this time.
I’ve been a back-end developer, a support engineer, a tester, a consultant, a (terrible) front-end developer, and much more.
I’ve worked on proprietary software, and I’ve worked on open-source software.
I’ve worked in a large open plan floor, I’ve worked in cubicle bays with half-a-dozen people, and I’ve had my own office.
I’ve had roles that were fully based at Hursley. I’ve worked from other IBM offices in the UK. I’ve been based at customer sites for months. I’ve had overseas assignments. I’ve had roles that meant travelling to somewhere different every month.
I’ve worked in teams so small they all fit around my dining table for dinner. I’ve worked in teams so large that we needed several coaches for the team social trip to London.
I’ve worked in distributed teams with team members around the world in four different time zones. I’ve worked in teams where we were all in the same office together.
I’ve worked on software that was first released in the 1990s, and I’ve worked on the first releases of brand new products.
The point I’m making… it hasn’t felt like the same job for eighteen years.
August 2003 – June 2007
IBM MQ (although it was called WebSphere MQ back then)
Developer
This is where I started learning how to be a developer. I learned Computer Science at university, but that wasn’t the same thing. Working in MQ taught me about creating software at scale. The importance of testing. How to make code that is maintainable. How to handle a critical customer outage and produce an urgent fix in a hurry.
They did a graduate rotation thing which meant I worked in different teams in my first few years – in Development, in System Test, and in “Level 3” Service. MQ was a huge organisation – and I got to try out lots of bits of it. It was a fantastic all-around intro to every aspect of software development.
I really threw myself into MQ as a space. I started an MQ developer’s blog and tried to convince colleagues to post to it. I wrote supporting apps and tools for MQ administrators in my spare time – some got released as unsupported “SupportPacs” on ibm.com and one ended up getting formally productised.
(Yes, I was a weird workaholic – get used to this, it’s a common theme)
June 2007 – August 2008
WebSphere Process Server on z/OS mainframes
“Level 3” Service Engineer
I’d not done anything with z/OS before, so this was a new platform to me as well as a new product. I never really loved either, which was a shame. But I’m glad I tried it: the team were lovely, and I got some great opportunities – including my first overseas business trip, travelling to the Philippines, Malaysia, and Singapore.
There was another unexpected benefit, too. Not being obsessed with the tech in my day job meant my evening/weekend coding went in new directions. I built all sorts of random things – an app to display your home energy usage, code to string random sensors together over MQTT, I got really into mobile app development (although this was before the iPhone SDK was a thing, so mobile app development looked different back then), and so much more. All of this ended up getting me my next job – in a way I’m not sure would’ve happened if I was still in MQ.
August 2008 – June 2011
Emerging Technology Services (ETS)
Emerging Technologies Specialist
Emerging Technology Services was a team that built prototypes and proofs of concept for customers in new and emerging technologies. The goal was to bridge the gap between Research and Development – exploring technologies that weren’t new enough to legitimately be “research” and weren’t yet mainstream enough to be in product development. We explored these emerging technologies with customers by showing them how they might be applied to solve their business problems.
I initially joined to do mobile and embedded software development, and I did get to do a lot of that. But I also started learning about AI and machine learning (something I’m still interested in today) with a particular focus on text analytics and natural language processing.
I got to play with so many other things too. I would learn just enough about some new technology or research project to be able to hack together a demo, and I used that to explain the future potential. Some of my projects were silly and daft, but they helped tell a story or start a conversation – about the implications of sensors, or social media, or analytics, or biometrics, or NLP, or so many other things.
In my more smug moments, I can think of predictions I gave in demos and presentations to skeptical customers that have since come true… I am probably conveniently forgetting more stupid predictions that didn’t pan out.
Each project would be in a different programming language to the last. I got good at figuring out how to hack stuff together quickly. Most projects I did were short – sometimes a week or two, sometimes a month or two.
I got to travel a lot. Some was glamorous and exciting – I went to all sorts of countries to build things. I also spent a few months in Farnborough, so I guess it balances out.
In hindsight, I can’t believe I was only in ETS for a few years, because it feels like I was there for so much longer. It was a lot of fun, building ridiculous things with some of the smartest people I’ve ever worked with.
And it put me in just the right place at the right time to get my next job.
April 2011 – April 2012
IBM Watson (the Research project that played in the gameshow)
Developer
February 2011 was the big demo of IBM Watson on a US TV gameshow – the result of a 4+ year research project. In the next couple of months, a team was formed to take that research project, and turn it into something that could be used to solve real-world problems.
The initial team was formed from engineers in ETS – so although I was still technically in ETS, I moved to work full-time on the Watson commercialisation project.
It was such an exciting, inspirational time. It was a genuinely impressive Research project. And there was so much for us to do: to improve the scalability, performance, security, manageability, and robustness of the code. None of those are necessary for a Research demo, but they were essential for us to be able to use it for more than a quiz show. Even just understanding the researchers’ amazing work was complex enough to start with. Those first few months were really drinking from a firehose.
April 2012 – December 2014
IBM Watson (the enterprise platform used in large-scale healthcare projects)
Developer
A new division was formed around IBM Watson. The ETS task (bridging the gap between Research and Development) was done. It was time for the ETS commercialization team to hand over the new productized Watson code base to the new division. But I didn’t want to. I was enjoying working on this stuff too much and didn’t want to hand it over and move on. It wasn’t just me – a few of us asked to stay on, and we followed the code over to the new Watson Division.
The goal was to build a platform that could underpin the healthcare projects that were being started using Watson.
I started focusing particularly on accuracy analysis: how we understood why the Watson system returned the answers that it did, and how to improve performance. I became responsible for the tooling that was used to perform this analysis. On the one hand, this meant supporting algorithms developers with tooling to evaluate algorithms and feature scorers they were developing. On the other hand, it meant supporting IBM Solutions teams delivering oncology projects with tooling and methodologies for evaluating training they were doing.
It was almost all Java development. I remember a lot of OSGi-related swearing.
My team was all based in the US now. That’s more common nowadays, particularly post-COVID. At the time it felt unusual – to me, at least. I got used to very quiet mornings followed by a scrum call in the afternoon when my team all woke up.
I did get to join my team in person from time to time, though – including one assignment in Littleton, MA for a few months where I was able to bring my family with me. They got to explore Boston while I worked over a long summer.
January 2015 – December 2016
IBM Watson (some of the cloud developer APIs)
Lead Developer
Time for another new organisation with the name “Watson”. Until now, Watson was just for huge IBM teams using dedicated server clusters for large long-term projects staffed by experts. But now the focus was on “bringing Watson to the cloud”: creating cloud APIs that developers without a background in ML could easily embed in their applications.
Everything needed to be reimagined to be cloud-native.
After a couple of years of being a (mostly) happy Java developer, I was thrust into the world of Node.js and microservices architectures. And I loved it.
We created an explosion of Watson services – dozens of them at one point. Watson wasn’t just about question answering anymore. I worked on a few, like Watson Engagement Advisor, Watson Language Translation, Watson Retrieve and Rank, and Watson Compare and Comply. Some were new. Some involved taking projects from Research and working out how to transition them into productised cloud services.
I was still interested in accuracy analysis and methodologies for training ML systems. You can see echoes of my earlier Watson Core work in what I started building for the cloud. Instead of building complex tooling and training expert users on how to use it, I created tooling to guide users through the process. I’m still really proud of some of what we created there – such as the task-based approach we created for Retrieve and Rank to drive users through a formal training methodology (without them needing to understand the ML principles behind it).
An early Retrieve and Rank demo
I helped with high-profile early adopters of the cloud APIs. One highlight was getting to help the Watson work that went into Wimbledon, which included getting to visit Wimbledon during the 2015 tournament. I’m not a big sports fan, but that was a great day.
January 2017 – December 2017
Managed Event Streams
Lead Developer
Time for a change. I went back to working in IBM Integration (the bit of IBM where MQ lived, where I had started out back in 2003), but this time to create something new.
“Managed Event Streams” was an idea: take API management (methodologies like describing APIs using OpenAPI, technologies like API Gateways, and capabilities like self-service developer portals for discovering and getting access to APIs) and bring all of that to the world of asynchronous protocols like MQ, MQTT, and Kafka.
I had a small team of a few developers and designers to help me flesh that idea out. We worked with customers to understand if this was something anyone would really need or want, and we built enough of a prototype to prove it was technically feasible.
It was a chance to work with a super creative team who really believed in this new idea. And after years of working in distributed teams across multiple timezones, it was lovely to work in a team in a single office again. We kept almost-but-not-quite getting the project funded – and spent the year building ever-more-refined and detailed prototypes and project plans.
There were highlights – it was my first chance to go to IBM Think (our annual customer event) held at the time in Las Vegas. Vegas is a weird place that I simultaneously loved and was appalled by in equal measure.
Although our plans and prototypes were amazingly well-received by the customers we worked with, we didn’t translate that enthusiasm into getting the project off the ground. It was shelved in favour of starting what I ended up doing next.
January 2018 – July 2021
Event Streams
Technical Lead (2018 – 2020), Architect (2020 – 2021)
We created a new product. Event Streams was Apache Kafka with the extra stuff around it you need when you’re running it in a production enterprise environment. And it was based on a focus on design and user experience, with the goal of making things easy for companies without a lot of Kafka expertise.
Kafka was pretty much new to me. And it’s a complex beast to get your head around. Every other part of what we were using was new to me, too: Docker, Kubernetes, Helm, Operators, OpenShift, and so much more. I’ve learned a lot in the last few years.
Not just on the technical side. My previous lead developer roles involved informally coordinating small groups of developers I thought of as my peers. Now I was the technical lead, for the largest team I’d worked in since I was in MQ (about thirty devs and designers at our peak). I learned a lot about technical leadership – partly from being coached by our technical architect (until he moved on, when I stepped into his role) but it also largely came from finding my own ways to do it badly in an impressive range of stupid ways.
I was able to grow an awesome team with an amazing dedication to improving the user experience. I’ve never seen a team of engineers and designers collaborate so seamlessly. It made our engineering decisions better, and it was just fun to be a part of.
There’s some overlap next. I kept this role while picking up my next project.
January 2021 – July 2021
Event Endpoint Management
Architect
I finally got to build and release the Managed Event Streams project I worked on in 2017.
We ended up calling it “Event Endpoint Management”. And we focused on Kafka as the asynchronous protocol to start with. But the core idea is the same Managed Event Streams that I spent a year pushing for and it’s finally a thing that customers can buy and use. I’m really glad it happened and that I got to see this through.
A demo of Event Endpoint Management
August 2021 – ?
IBM Technology Garage Client Engineering
That brings us to today. It’s time for another change.
On my 18th anniversary at IBM, I’m starting another new role in another new team. This post is already so much longer than I’d planned, so I’ll talk more about this another time!
Edit: more about Client Engineering here!
Tags: career