Thoughts from JUXT's XT26 Conference - Part 1

 This blog will my thoughts from JUXT's XT 26 conference: https://www.juxt.pro/xt26/

 

Summary:

• We are mid storm - AI is changing the industry.
• No one knows what's going to happen, if they claim to know, they are just trying to sell you something.
• We were promised that AI would make us 10x more efficient - it's not.
• We have shifted the bottleneck from coding to PR reviews. 
• You can out-source your thinking but you can't outsource your understanding 

 

Intro 

 After a hectic morning I managed to just catch Jon's intro which explained we are "mid storm" given the changes AI is making to the industry.

 

 

 

Martin Trojer - Meta - Surviving the Claudepocalypse

Martin's talk was great, so many other speakers referenced it throughout the day.
 

• Something has changed
• We all get whiplash at the rate of change.
• We are surprised how well these models work - we all get vertigo every now and again.

Tackle the big questions: "what does it mean for us as devs, what is left for us humans to do… philiosphical".

Answer: no one knows yet what it means.

Main implication: New expectations: prototyping, refactoring way quicker. BUT - goal posts havent moved - to feel good now, is to leverage the tools to keep up with expectations.

Martin then described three different levels of code.

1. Plausible code: looks like the real thing, but breaks down. 
1. In the old days to arrive at this state, a lot of effort would have gone into this level of code. You'd assume a lot of heartache has gone into getting to this state. Using your old eyes, looks good.
2. It's likely not ready to be shipped.
2. Trusted code - human has spent more effort on it, human has read it and kicked around the code. Surprising bugs ironed out. This stage: human can say "we can merge"
3. Proven code…. Code that survived contact with reality - this is the goal for all of us. This is still really hard. Not clear that this has become cheaper - even with coding agents.

Using your old eyes shipping plausible code is a BIG RISK… teams keep shipping while understanding less.

Velocity goes up, understanding goes down and down which is very dangerous!

Delayed tax… benefit up front, but the bill comes later when the team loses the plot on why the code does what it does.

Cognitive debt-drowning in pull requests. Feeling like more effort in reviewing than producing.

Leads to bad things: quality of reviews goes down.

 

The effort is shifting from coding to understanding. It takes 5 minutes to create 10 pull requests, it then takes 10 hours reviewing them!

Human context lost.

Lose the human oracles that know where everything is.

Context is lost.

Practical survival guide:

• Dive in.
• We (at Meta) have a distribution of devs on AI. Enthusiasts and skeptics.
• Refusing the tool is not a winning strategy!
• The genie will not go back in the bottle.
• Your job to find out how it fits into your workflow to solve your problems. 
• Must dive in and use the tools.

 

Boring work first

• Start with boring work - low risk-change
• Easy to review, failure obvious
• Low risk != low value
• Agents shine here. Agent is good at, but also easy for you to evaluate the output… should be obvious when it does something silly.
• The old gnarly tickets.
• Release your inner hoarder

Don't be a hoarder of knowledge - write it down for AI to use. 

 

For example "How we do DB migrations... usually a couple of dudes who know how to do it", but with agents, super useful to write it down.


3rd act - Philosophical…

 

Survival tips

• you can't check out, you have to be intellectually and emotionally engage
• Don't ship plausible code.
• Interesting theory.... Pain is your superpower - Agents don't feel pain... Devs do.
• Exceptional devs - they feel the pain accutely - they feel personally offended by a bad PR.
• Lean into this.
• It's an important signal we shouldn't ignore - it tells you something.
• Agents - Happy to keep adding more if statements.
• If you don't understand the actual problem, you cannot come up with the right abstractions

Quote from (someone?)... "You can outsource your thinking, but you can't outsource your understanding"

So who survives the Claudepocalypse?

• What is left? 
• To understand what we are doing.
• Who survives?
• The person who lives up to the new expectations.
• Being able to explain what the system does and why matters.
• You will say no a lot with AI "that doesn't make sense!"
• You are the professional that keeps the understanding.

Q&A to the group

• Question: Are these survivial things a function of experience? If so - what happens to the new joiners?
• Answer: New employees will learn in a very different way - how do they develop their spidey sense to know stuff is wrong.
• Question: Experienced devs used to enjoy the dopamine hit when you crack a problem with code. How do you tackle that now?
• Answer: Some devs like the editing, for me and the people who have excelled. I derive the joy of building stuff that works. I feel empowered.

Martintrojer.github.io - slides.

Simone Steel - Data Cartography: From static blueprints to navigable maps

Simone's talk focused on the goal of trying to come to a shared understanding of data at all levels of an organisation based on "data maps" i.e. Data Cartography.

Miss-understandings… Simone argued that there is no shared picture of the data landscape. Pushing automation at a faster pace than we can explain where the data pitfalls live. We are facing cognitve debt with data - copies of data etc etc Losing ability to explain the entire system. Relying heavily on catalgoues…


The Audience problem
There is only one data domain but multiple representations that are not understood by everyone.  

 

Simone argued that everyone understands maps, so why not use this visual grammar.

Key concept - one map, just different levels of zooming in and out.

 

 

Will Bassett - CIO Equities and Cross Asset Financing Technology -  All in Equities Technology: The promise is real, so are the problems.

Jon Pither introduced Will and said that challenge of modernising a current platform can feel like "rebuilding the engines of a plane while it's still flying".

Will made the following points in his talk:

• The promise is real, so are the problems
• We are at an inflection point in software development
• AI is improving 
• Dev productivity, time to market, cost potential (theoretical case is compelling), tackling the backlog, a new kind of democratisation.
• Arithmetic is very attractive - if running a team of 1200 engineers.
• Front office users can now build working prototypes themselves
• Instead of a story diluted through a product owner.
• AI doesn't create new risks - it exposes and explodes existing ones.

 
Cost Control - Tokenomics

• How do we account for the number of tokens.
• Tokens don't behave like seat licenses.
• Not every task needs the most frontier model to perform.
• Anthropic - going public. Once answerable to public markets they can't continue with the artifiically low cost to us… they just want to grow right now!
• Question is: How do we manage this? - Treat it like platform cost… or business PnL to be managed.

Proliferation of code & duplication

Will asked "Who has discovered that two teams built the same thing" - lots of people raised there hands, Will argued it's now more likely than before.

Three teams, three versions of the same tool.

Central Governance: a necessary evil in a regulated industry.

Not a fan of big centralised decissions - need to push it to the team coding.

"We just trust people" doesn’t fly with regulators, so we need "just enough".


Cyber & Patching - Discovery is accelerating, patching isn't.

• Mythos not made public - found dormant flaws.
• We need to hire the skills for this - they don't exist yet.
• Uncomfortable truth: Most orgs are ahead on capability and behind on controls.

Shadow AI

• Very powerful tools in people's pockets/bags. 
• If you don’t give access - people use the stuff in their pockets. "If something goes wrong, who can reconstruct what AI had access to"

Role Impact

• BAs, Orchestration 
• From doing to directing - Institutional knowledge is vital.
• Rather than being the thing that produces, you are orchestrating.

Graduate roles
 

The entry rung is disappearing…. Is the process defined for a world that no longer exists?

Close to the Metal - Modern Low-Latency Development - Tom Dellmann - Chronicle

Introduction: This talk will have no AI (to which many people cheered!). 1ms is a long time for Tom

 

Tom made the following points

• Averages can be deceiving
• We need to see latency histogram/profile (with percentiles)
• Java can be highly tuned to give low latency performance.
• Finding the balance between readable/easy to maintain vs performance but you can have your cake and eat it too.
• The impact of data layout and garbage collection is very significant
• Low latency apps - we want to avoid GC entirely - They also avoid usage of the new keyword in Java (this astounded me!)
• Chronicle try not to write multi-threaded apps and have everything on an event loop. This gets rid of the scheduling issues.
• Stopping threads from wandering - Linux Kernal can be helpful to try and stop things happenning to your threads. The point is, your thread might be migrated.
• CPU isolation - must isolate it in Linux so other Linux users don’t steal it.

 Q&A

•  "Why not use another languages like Rust, C?" 
• Answer: All about skilled devs. Hard to find devs interested in this. C++ I find a very big language and challenging to work with on the tool chain.

My thought: Why can't we just write more normal looking code, use the new word and kill the app when it comes close to needing garbage collection and start another? 

 

 

Spectroscopy for Software: Measuring Legacy Code with AI  Henry Garner | JUXT | CTO 

 

Henry started with the question "What makes legacy code legacy? Is it code without tests, code we didn't write?" - By this definition - we are writing more legacy code with AI.
 

Mythos found lots of bugs but Henry was fascinated by what bugs it did not find. Bugs at the higher more behavioural layer e.g. adding two different currencies? 

 

Developers are good at reconstructing the intention. However, if we can provide a higher level of representation - AI can find these behavioural bugs.

Allium spec

• Represents behaviour at domain level.
• Encodes it in nouns and verbs that are recogniseable.
• Given, rule, when, requires
• Extract specifications in this format - no idea of language

What if we ran Allium on the oldest code base we could find....

• Margaret Hamilton - coined the term software engineering
• Eyeball - they found bugs by reading the code line by line.
• A lot of the Apollo lander code has been posted on github now.

Allium found a bug in the code.

• No release of lock from gyro
• Reproducible on an emulator.
• Fuzz reproduced the bug.
• It turns out it was found in 1970 and Fixed on Apollo 15.

"Add a new feature to tilt the lander towards the moon so the crew can take a photo outside of the window"

 

Claude: "Sure... done" - It just assumed many points

Allium: 

• "Which window, there are five windows",
• "What about the gyro locks, should they be released before of after the tilt"

 

Henry's talk was truly fascinating (this might be because I'm a huge fan of the moon landings!)