Thoughts from API Days London 2024 - Day 2

Green IO Opening Remarks - Gaël Duez Founder - Green IO

Gaël started the Green IO conference (this was a track within the APIDays conference) with a very welcoming and friendly tone that felt quite different to other conferences. He encouraged us to greet our neighbour and say hello which went really well. 
The room was completely full with many people standing at the back. I can imagine that next year this conference will be moved into a bigger room. 

Gaël Duez Kicks off Green IO with a friendly welcome and a great diagram showing people working together to pull down the line graph of CO2 emissions
Gaël Duez Kicks off Green IO


Keynote: Consensus As A Climate Lever Room: Commercial 3 Chris Adams Executive Director - Green Web Foundation


Chris explained to us "just how messy carbon calculation can be". My main takeaway from Chris's talk was exploring the concept of consensus and how having too strict a definition can be counter-productive. Instead people now realise that to get things done we need to instead aim for "rough consensus" and came up with a great quote...

"We reject: kings, presidents, and voting. We believe in: rough consensus and running code." David D. Clark 1992

David D Clark was instrumental in establishing the TCP IP protocol which is now ubiquitous. This philosophy of rough consensus has inspired many within the green tech movement.

An image showing Chris Adams in front of a slide with a great quote on rough consensus from Killian Daly - executive directory of Energy Tag

This was very inspirational and can certainly be applied to any work environment!

Chris finished with a mention of the domain.com/carbon.txt proposal. This simple url would allow sites to provide this metadata relating to the carbon emissions. Search engines could then highlight the green credentials of a site on the search results page. 


The Top 10 Green Software Tools on GitHub Room: Commercial 3 Paull Young Head of Sustainability - GitHub

Paull Young walked us through the top ten green software tools on github from this list: https://github.com/github/GreenSoftwareDirectory 


  • Scaphadre: is a metrology agent dedicated to electric power and energy consumption metrics. Use it to measure the power consumption of your tech services and get this data in a convenient form, sending it through any monitoring or data analysis toolchain.
  • Kepler(Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe performance counters and other system stats, uses ML models to estimate workload energy consumption based on these stats, and then exports them as Prometheus metrics. Use it to learn about energy consumption of Kubernetes components such as Pods and Nodes.
  • CO2.js An open-source JavaScript library that enables you to estimate the carbon emissions produced by transferring bytes of data on the internet, get different forms of grid intensity data such as annual average and marginal data by country, and make automated queries against Green Web Foundation’s Green Domain’s dataset.
  • Kernel_tuner Create optimized GPU applications in any mainstream GPU programming language (CUDA, HIP, OpenCL, OpenACC).

Electricity Maps - Not a top ten GitHub project, but a cool site to be shared!

Renovate to Innovate: Fundamentals of Transforming Legacy Architecture Rashmi Venugopal Senior Software Engineer - Netflix


Rashmi started her talk by looking at various different growth diagrams. The classic is the hockey stick that shows exponential growth over time. Rashmi argued that any set of systems that go through this growth are destined to become legacy and need to be re-written.

However, these are good problems to have as they are a bi-product of success!  

Definition of legacy? An app is one that is no longer able to be extended to meet the business demands.

Legacy apps are also associated with:
  • a decrease in velocity (developers find them hard to work on). This is due to...
  • increased operational complexity
  • cognitive complexity - Some of your institution will likely know the history and many nuances of how it works.
How did it become legacy?
In real world software is affected by all.

Technical Renovation vs Refactoring

Rashmi had a great analogy here...

Refactoring - Is like organising my closet - moving things around so I can access my clothes, maybe get rid of stuff too.

Technical renovation - This is where we break the walls down. It goes beyond moving things around. Gives you capabilities you didn't have to begin with. Refactoring is still valuable, however technical renovation is bigger and done less often.

Strategies

  • Make it work, make it right, make it fast. Technical renovation is a big task. Working out what to do first is hard. Prioritise making it work (i.e. production stability). You're out of business if job cant be done.
  • Make it right - don't over engineer.
  • Make it fast - prioritise performance.
  • Evolutionary Architecture
    • Incremental changes. Big bang always goes wrong.
  • Deprecation driven development - what do we gain by deprecating (i.e. switching off functionality) - Sometimes you cant justify the cost.  Netflix dropped support for dvd.com. It became very difficult to fund and hard to keep it working for users.
  • Need a growth mindest! These are aspirational. No silver bulletts.
It will be bumpy/thorny!


Unlock Peak Platform Engineering Excellence with API Gateways - Alessandro Vozza Founder and CNCF Ambassador - Kubespaces.io

Alessandro was fun to listen to. He loves the word love and he loves to keep in mind the user we are building systems for.

Regarding evolution of design, he showed an image highlighting the evolution of the SpaceX Raptor engine through versions one to three.  Version 3 looked super simple and way cleaner without the extra external pipes,


Microservices Paradise: Is it possible to achieve? Irina Kamalova Founder of Women Coding Community - Senior Software Engineer at Revolut


Irina started asking for a show of hands for who had seen "Microservice hell" and also who believes if there is a "Microservice Paradise". Only a few people thought paradise was possible which surprised me! I thought more would be optimistic!

Here are Irina's points

1. Design - Strict rules
Keep distinct layers separate in your stack.

Front end -> Flow -> Stateful

Don't allow the arrows to go backwards or any other combination otherwise you end up in a mess.


2. Follow good names.
Names should follow business domains e.g. Retail, Business, Trading.
Don't go for abstract names such as Saturn Elephant, Vanilla.

3. System design review expert committee
It's good to review big decisions

4. - Introduce the risk assessment (for changes to design)

Complexity metrics - How long will it take and how many developers are needed? Has this ever been done before?
Materiality - How many users will this change affect?


Building and Maintaining API Platform @ Enterprise Scale Kailash Sahoo, Saranya Balaji, Angelo Wolf - Lloyds Banking Group

Kailash introduced their talk by comparing airport complexity and operations to Microservice operations. Lloyds have some impressive numbers with regards to their re-architecture. I didn't get too much from the talk.


The Web Sustainability Guidelines Alexander Dawson Sustainability Group Co-Chair - W3C


Alexander is co-chair of the W3C Sustainable Web Community Group SWCG and editor of the Web Sustainability Guidelines (WSG). SWCG is in the early stages of becoming an official W3C group and will hopefully be incorporated soon. 

The web sustainability guidelines have been created over three years and are still in development. 

Some points:

  • Web sustainability benefits people as well as the planet and it uses many practices were already aware of.
  • Reduces risk of green washing claims, negative publicity or legal non-compliance.
  • Governments and international bodies are addressing digital impacts.
  • Green claims directive. Legislative that when you claim to be sustainable you can actually back it up.
  • WSGs - many other guidelines that have been pulled into one.

The guidelines are massive. Ninety four guidelines, two hundred and fifty success criteria, over six hundred pages and over one thousand five hundred pages. Alexander's advice is Don't panic! If you can do something then that's great.

Quick reference and checklist: https://w3c.github.io/sustyweb/

Key takeaways from me:

  • Alexander was asked "Are you worried that big tech will block this when it becomes more official?". Alex said that there are already big tech firms contributing and they have been "nothing but helpful". However, if resistance then they are "up for the fight" to which people cheered!
  • Digital industry is now responsible for 2-5% of global emissions. This staggered me!

Beyond User-Centricity - Designing Sustainable Digital Products Thorsten Jonas Digital Sustainability Trailblazer and Founder - SUX - The Sustainable UX Network

Thorsten was incredibly engaging, he had a ton of passion and knowledge he was just desperate to tell us about. Other speakers commented that he would be a very tough act to follow! On top of all this, Thorsten even paused his presentation when he realised I was just a second too slow to photograph one of his slides and was kind enough to go back :)

Thorsten made the point that whatever we optimise for, if we forget the other factors, then someone or something pays a price.  See the image below which illustrates these trade-offs

Thorsten shows the tradeoff with his slide showing the balance between Ecological, Societal, Business, User and Human


This slide references the following page: https://www.smashingmagazine.com/2022/10/sustainable-web-development-strategies-organization/ 


  • On a single page, the "Fact Sheets page" (presumably somewhere on smashingmagazine.com - though I was unable to find it).
  • They can save 85% of images’ file sizes with compression without a noticeable loss of image quality. 
  • Assuming 1,300,000 annual page views…this is:
    • 5.2 Terabyte of wasted traffic.
    • This costs EUR 1000–1650 
    • 17.28 tons of CO2
      • which requires 925 trees to be planted to offset

I remember back in 2005 working at rightmove.co.uk and spending time compressing images on the homepage to ensure they were as small as possible without impacting the visuals. This was motivated by page loading time, however we were also saving CO2 without even considering it!


Stress Testing Green Code in the Wild - Tereze Gaile Strategic Client Architect - Salesforce

Tereze has worked with Salesforce client Astra Zeneca to measure and reduce their CO2 emissions. 
My key takeaways of her talk was the need to collaborate. She really started making progress when she was able to get time from engineers to help her answer the questions she had.


Key Learnings on Day 2

  • I came to API Days without a single thought on sustainable tech. However, the Green IO conference has really inspired me and got me thinking. I remember at the beginning of my career working hard to compress images on the rightmove.co.uk homepage so as to save bandwidth. Now people have video backgrounds on their sites! It seems we have gone too far with our data and we need to steer back to our old lean ways of treating data as a CO2 gobbling commodity to be minimised!
  • Rough/broad consensus and not total consensus allows you to get things done!
  • Technical renovation doesn't mean you did things wrong on day 1!



Comments

Post a Comment

Popular posts from this blog

Lessons learned from a connection leak in production

We recently encountered a connection leak with one of our services in production.  Here's what happened, what we did and the lessons learned... Tests and Monitoring detected the issue (silently) Our automated tests (which run in production for this service) started failing soon after the incident occurred.  We didn't immediately realise since they just went red on the dashboard and we just carried on about our business.  Our monitoring detected the issue immediately and also went red but crucially didn't email us due to an environment specific config issue. The alarm is raised A tester on the team realises that the tests had been failing for some time (some time being 4 hours... eek!) and gets us to start investigating. Myself and another dev quickly looked into what was going on with our poorly  dropwizard  app.  We could see from the monitoring dashboard that the application's health check page was in error.  Viewing the page showed us that a downstream ser
Read more

How to connect your docker container to a service on the parent host

Recently I needed a docker container to access a service running on the parent host.  A lot of the examples on the internet were very close to what I needed, but not quite. First off, docker's documentation provided an example that (as far as I can see) doesn't actually work.  The following (I think) is meant to inject the src ip address of the parent host's default interface: $ alias hostip="ip route show 0.0.0.0/0 | grep -Eo 'via \S+' | awk '{ print \$2 }'" $ docker run  --add-host=docker:$(hostip) --rm -it debian What it actually does is inject the ip of the default gateway on the parent host which doesn't help us. Michael Hamrah 's blog details the " gateway approach " of getting the parent's host ip from within the docker container.  This works great, although I wanted to inject the ip on container run and not have that logic within my docker image. Inject the src ip of the docker interface I think the best
Read more

Client Side vs Server Side Session

Image
We recently looked at replacing our legacy session management system with a new one.  During this analysis, we came close to choosing a client side session but eventually concluded server side was better for us.  Here's why... Client side session In this model, all session state is stored in the client in a cookie.  The benefits of this are you don't need to worry about persisting and replicating state across nodes, session validation is lightning fast since you don't need to query any data store which means it's super scalable.  The session cookie must obviously be tamper proof (to prevent people creating a session of their choice) which is achieved by signing the cookie using asymmetric cryptography. The signing of a cookie value uses the private key, the validation uses the corresponding public key.  Our idea was to try and keep the private key as private as possible by storing it in memory only.  Each node (4 shown below) would create a new private and public
Read more