Thoughts from JUXT's XT24 Conference

I was fortunate enough to attend and speak at XT24.  Here are my notes and thoughts from the event.

Fran Bennett - Interim Director & Board Member - AI: UNINTENDED CONSEQUENCES

Fran made some great points regarding the risks of trusting AI too much.  While some people are worried about super intelligent robots, Fran points out a far more likely set of "unintended consequences" that have already started to materialise.

How do you test a system is working?  You need an acceptance criteria.  Developing a set of requirements is a hard to thing to do and it's easier to just... not specify them.  If that's the case, how can you test your system is working?

Fran made the comparison of AI products with the recent British Post Office scandal where people blindly assume "The system is infallible".  Clearly IT systems are not, real life is messy!  We must always keep in mind that the system could fail in weird ways.

Human testing AI output also has issues.  It may seem counter-intuitive, but users are better at reviewing the output of a more unreliable system than a reliable one.  If a system only fails very rarely, the users start trusting the output and not actually reviewing resulting in errors going unnoticed.  AI can fail is some very weird ways (we saw some images of fruit that at first glance look perfect, but on closer inspection are completely wrong).

Fran's recommendations for AI regulation make perfect sense when you consider the "real world" equivalents:

  • Test safety in a crash - as we do with cars
  • Assign liability for harm - as we do with food poisoning
  • Hear voices of those affected - as we do with planning permission
  • Right to contest decisions and request explanations - as we do with human bureaucracies
A great talk!


Allen Rohner - BREAKING THE BANK WITH CONTRACT TESTING - CTO & Founder - Griffin

Allen's talk focused on how to fully remove test flakiness by making them pure and deterministic. 

By the sounds of it, Allen has achieved this at Griffin bank and gone to extraordinary levels and efforts here. Not content with removing network level faults, it sounds like they have also managed to even remove threading/race-condition flakiness too by running the entire cluster on a single process.

The goal is to isolate all side effects and non-determinism.


Me - WORKFLOWS, THE BENEFITS & PITFALLS

I spoke about my experience leading a team which built a workflow based orchestration system.
Huge thank you to JUXT for having me!

DATA COLLABORATION AND PRICE DISCOVERY FOR THE COMMODITIES MARKET - Sebastian Ferraccù | CEO & Co-founder | Artis.Works

This started with a demo of an impressive UI showing a constantly updating table of prices. The scale of the data was rather colossal (though I lost the exact numbers in my notes).
The UI gets all of it's data from the server from a single point of truth.  

The UI displays fields that require calculations to be performed on a huge web of dependencies.  These were visualized by a graph which looked simple at first until the view gradually zoomed out more and more to reveal a giant web of dependencies necessary for the calculations.  

For some reason this reminded me of the javascript library knockout js which keeps track of which fields need to be updated based on their calculation methods.  I wasn't sure what library was used for these calculation at Artis.Works.  However - the scale of it was seriously impressive.

TALES FROM THE BUY-SIDE - Kim Johannessen | CTO | Consulting & Advisory

Kim has had a long career and many roles at various institutions on both the buy side and the sell side.  This fireside chat was to discuss the main differences between the two.

Buy side is any company that purchase securities such as investment managers, pension funds, and hedge funds. The sell side are companies that issue, sell, or trade securities.  These are typically investment banks.

Kim told us that we could expect the buy side to move a lot faster than the sell side.  In Kim's experience the sell side is siloed in both technology and function.

Developers in the buy side have to be a "jack of all trades" where you might help many areas of the business such as Risk, Trading, Research or HR.  There is typically a flat structure.  He also warned that there are "lots of egos in a hedge fund".

Sell side typically move a little slower and spend more time in analysis.  They typically always take a long term view where as the buy side just has investors and everything is needed "yesterday".

Kim also explained that it used to be very easy to setup a buy side firm.  Nowadays it costs millions due to the very high "barrier to entry", specifically related to compliance.  Hedge fund can be very lucrative however due to them typically charging 2% on all investments as a management fee and 20% profit fee.

Someone asked about the benefits of using IT services such as Aladdin by Black Rock.  Kim said it was "unpalatable" in terms of costs since the vendor wants a percentage of the profits.  Buy side firms are sometimes run very lean with some CTOs sometimes having to pay AWS bills on personal credit cars to keep services up due to not having enough money in the company account!

XTDB - Malcolm Sparks & Jeremy Taylor - JUXT

I had heard a bit about XTDB before the conference, but this was the first time I had seen it explained in some detail.  Essentially XTDB is a "Bitemporal database" which means (I think) means it stores two dates for each entry:
  • The date the event/thing happened
  • The date the event/thing was known to the system.
This is great as it means systems can re-run reports before some corrections were inserted into the system.  It turns out many systems have fixed time windows when they do not allow any new records to be added/modified to allow some batch run to finalise a report.  This is due to the inability to separate these dates!  Jeremy Taylor showed us feature by showing a slider navigating forward and back through time.  

During this talk we also learnt that SQL is now 50 years old!  We celebrated this at the rooftop bar at the end of the day.


WE NEED TO TALK ABOUT JSON - Oliver Hine - JUXT

This talk was a lot of fun due to Oliver's great presentational style.  Oliver went to the trouble of creating a Star Wars style introduction which summarised how JSON had won the war, despite lacking features such as comments, dates, times uuids and sets.  We just have to shove dates and times into strings and accept broken portability - which we (as an industry) have kind of decided is fine.

However JSON Schema to the rescue!  Oliver showed how JSON schema (with some additional layers) could be used to build some complex UIs which are actually good!  I saw this product first hand and can definitely say it was a huge success.  They were able to add new features very quickly and the UI/UX was great too.


SIMPLIFYING ALL THINGS MONEY: LESSONS LEARNT BUILDING A BANK FROM SCRATCH - Vlad Yatsenko - CTO & Co Founder Revolut

Vlad was asked many interesting questions from Jon about the IT setup at Revolut and how they got started.  Some notes:
  • When Revolut started, they only had two interns on customer support
  • They use in-house frameworks and a very standardized architecture.
  • According to Vlad - If a developer proposes something that is different from the architecture, it's usually because they haven't fully understood the current setup.
  • A change from the architecture standard has to be a lot better otherwise it's not worth breaking the standard.
  • They don't use Spring
  • They use PostGres
  • Due to standard architecture they were able to migrate to kubernetes very easily (i.e. in a few days with just a few people)
  • Vlad recommends starting with a monolith (as others during the day recommended).
  • When too many people stepping on each others toes, then split!
  • Vlad's biggest regret was a data warehouse solution that took four years and couldn't scale.
The interesting thing about this talk was that it generated a lot of discussion in the rooftop bar later!  It also seemed to contrast with Zohar's views on his talk (more on that below).

FUTURE OF FINTECH - Expert Panel - Moderator Jason Bloomberg

Jason did a great job moderating this and injected a lot of energy to the debate.  

A few topics were discussed:

AI
Malcom made the comparison that AI feels like a tech cycle reminiscent of the early web (.com bubble).  Perhaps the same will happen and we'll have a crash with only a few survivors?

Fran made the point that a some companies are starting to run out of training data now and that their CTOs now have a different less bullish tone. There's also perhaps nervousness with people who dumped a lot of money into it.

Mark Burgess explained that "trust" is a big factor in that we need to feel good about the answer.

Vlad mentioned that the fear of us all being replaced seems to be unjustified.

Jason said that AI seems to be getting better and better at pretending.  I think others also mentioned that this is the reason AI is used so much by scammers!

What might go away between now and the next Juxt conference in 2028?

Mark: The mainframe!
Malcom: Passwords are n their way out

Cyber security - The bad guys only have to find one way in, the good guys have to close all of them!  Are we heading for dystopian nightmare or will CyberSecurity win?

Fran: Neither side will win, we'll go back and forth.

Jason: I have seen too many films where people get their eyes ripped out to trick a camera/eye-scanner.  This got a laugh!

Malcolm: The PIN put the liability on the user

My thoughts: I am confused as to why social login hasn't taken off.  Login with facebook/google etc etc seemed to be popular around 10 years ago - what happened?

Decentralized Finance/Crypto

Mark: Crypto currency feels like an app looking for a solution.  It's also unethical due to the amount of power it takes. We already have cash!




NOT SO CLEVER: HOW WE BUILT A GLOBAL-SCALE SUCCESSFUL RISK SYSTEM GUIDED BY SIMPLICITY - Zohar Melamed  


Zohar explained that his talk was not how to build a risk system, more the guiding principles that were used to build it.  Consider the following:
  • How easy is it to test as close as possible to production?
  • How clear is the demarcation between various parts of the system.?
  • How frequently can it be deployed?
  • How steep is the learning curve?
Other values:
  • Simplicity.
  • Avoiding "magic".
  • Promoting technical choice (here is the contrast to Vlad's talk above)
  • Evolution vs design
Where Zohar seems to agree with Vlad - start with a monolith and split later.  Zohar also said "If you're lucky, you'll start with a simple system which will end up complicated... because your system is useful and it delivers value for the business".

We should also all strive to optimise/improve anything that slows us down (e.g. chat apps you use).

Zohar's talk was a great way to wrap up the conference and produced a lot of discussion in the bar afterwards.

Comments

Post a Comment

Popular posts from this blog

Lessons learned from a connection leak in production

We recently encountered a connection leak with one of our services in production.  Here's what happened, what we did and the lessons learned... Tests and Monitoring detected the issue (silently) Our automated tests (which run in production for this service) started failing soon after the incident occurred.  We didn't immediately realise since they just went red on the dashboard and we just carried on about our business.  Our monitoring detected the issue immediately and also went red but crucially didn't email us due to an environment specific config issue. The alarm is raised A tester on the team realises that the tests had been failing for some time (some time being 4 hours... eek!) and gets us to start investigating. Myself and another dev quickly looked into what was going on with our poorly  dropwizard  app.  We could see from the monitoring dashboard that the application's health check page was in error.  Viewing the page showed us that a downstream ser
Read more

How to connect your docker container to a service on the parent host

Recently I needed a docker container to access a service running on the parent host.  A lot of the examples on the internet were very close to what I needed, but not quite. First off, docker's documentation provided an example that (as far as I can see) doesn't actually work.  The following (I think) is meant to inject the src ip address of the parent host's default interface: $ alias hostip="ip route show 0.0.0.0/0 | grep -Eo 'via \S+' | awk '{ print \$2 }'" $ docker run  --add-host=docker:$(hostip) --rm -it debian What it actually does is inject the ip of the default gateway on the parent host which doesn't help us. Michael Hamrah 's blog details the " gateway approach " of getting the parent's host ip from within the docker container.  This works great, although I wanted to inject the ip on container run and not have that logic within my docker image. Inject the src ip of the docker interface I think the best
Read more

Client Side vs Server Side Session

Image
We recently looked at replacing our legacy session management system with a new one.  During this analysis, we came close to choosing a client side session but eventually concluded server side was better for us.  Here's why... Client side session In this model, all session state is stored in the client in a cookie.  The benefits of this are you don't need to worry about persisting and replicating state across nodes, session validation is lightning fast since you don't need to query any data store which means it's super scalable.  The session cookie must obviously be tamper proof (to prevent people creating a session of their choice) which is achieved by signing the cookie using asymmetric cryptography. The signing of a cookie value uses the private key, the validation uses the corresponding public key.  Our idea was to try and keep the private key as private as possible by storing it in memory only.  Each node (4 shown below) would create a new private and public
Read more