Thoughts from API Days London 2024 - Day 1
I was fortunate enough to attend API Days London today. Here are my thoughts from the various talks.
Welcome Breakfast Panel Discussion: The API State of the Market Report 2024 Edition
The room was almost completely full to host a panel discussion with:
- Mehdi Medjaoui Founder and Chairman of Apidays Conferences, Author of Continuous API Management - Apidays
- Mark Boyd Director - Platformable.
- Maurizio Garzelli - Sr Consultant, Solutions Architecture - Boomi
They showed a big image with 2500 icons of different vendor supplying different solutions. They said that eight years ago this was just aroud 150. The point was made that, although this looks complicated, all of these different vendors have the goal of simplifing things for the developers of APIs.
Mark made the point that multiple forces are driving the explosion of API tech providers. Firtly, social media, secondly sharing of data from devices being connected.
The point was also discussed regardg the "bundling / unbundling" cycle. It seems man vendors will come together and bundle their solutions into one. Then the reverse happens where vendors then unbundle and split their functionality into smaller solutions solving a narrow scoped problem.
I saw more evidence of the former (bundling) in the later talks - see next two talks below.
Another topic for discussion is the standardisation which reduces risk of vendor lock in - for eample OpenAPI Standard now becoming the defacto standard.
AI - This was discussed as a topic still in the "hype stage", though the hype seems to be dieing down according to the panel. An unanswered question is the environmental concern of AI due to the huge compute cost.
The panel suggested that secrity seems to be a topic going the bundled route.
Hands-on Apache APISIX - Navendu Pottekkat Developer Advocate and Zeping Bai Solutions Architect - API7.ai
Navendu started off by explaining what an API gateway does including brief summaries of user auth, throttling, canary releases, circuit breaking and directing traffic from /old to /new urls. He also summarised tracers, loggers and metrics.
The talk then moved on to a hands on demo of APISIX, an opens source Apache project: https://apisix.apache.org/ which I had never heard of before. APISIX is a "dynamic, real-time, high-performance API Gateway" which aims to provide "rich traffic management features such as load balancing, dynamic upstream, canary release, circuit breaking, authentication, observability, and more."
Assuming you have docker installed, an internet connection, it was very easy to follow the demo and get APISIX setup. This also assumes you were happy to copy paste some scripts form the internet which curl bash scripts to into the sh shell (I did look at them before I ran them - and I trusted our host!) it was very simple. Possibly more of a testament to docker (which I still love).
We managed to configure a route and also setup load balancing via some simple post requests from the command line. Full docs here: https://docs.api7.ai/apisix/getting-started/configure-routes The http requests were sent from APISIX to two remote services that are simply freely available APIs. httpbin.org:443 and mock.api7.ai:443. requests to the simple /hearders URL could be seen to be balanced between the two "nodes".
APISIX can also be extended with GO and Python plugins.
API7.AI offer many services ontop of API6. Their website suggests they offer a bundled product to solve all your API gateway needs.
Gavitee - Elesh Mistry - Lead Solutions Engineer
Quickly running from one room to the next, I missed the stat of the talk! It seems that Gravitee are selling a very simillar product to API7.ai.
Elesh Mistry showed us a demo of Gravitee which provides "Full API lifeccle managemet" with a particular focus on event based systems. They apparently "attract a lot of postal companies" due to them having an inherently event based domain.
Lot of customers, also banking customers including TIDE
Gravitee has full API lifecycle management.
Elesh made the point that working with Kafka is not simple and Gravitee can help with this. Gravitee also help bringing APIs from other gateways into one "unified place"
Elesh then described a problem domain I never would have imagined.
During covid, lots of industries were losing money and looking for new ways to get revenue in. How can you get data about NBA players into the market. The NBA started selling data on the health of athletes. Hear rate curves. Companies were then using that data to manipulate odds. Realtime data affecting the odds for gambling companies. - How do you sell the real time data. Can expose data to API?
Exposing event based data publicly as a service provider over REST is indeed a challenge! I didn't know until today that a lot of thought has gone into this. In fact an entire initiative exists to solve problems in this area: https://github.com/asyncapi
Elesh's Demo showed the Gravitee admin UI allowing developers to create Websocket based API that exposes data from a kaka cluster. Impressive stuff.
I asked Elesh: "As a developer who likes version control and build pipelines - how can I make sure the UI changes are deployed in same way?" He explained that the admin UI is rarely used by developers (used more for demos) and most people use build/deploy pipelines that utilise Gravitee's Admin APIs or via helm. UI is optional. Developers/Customer rarely use it for production usage.
Someone asked about "vendor lock in" and whether or not Gravitee is "following specs to avoid vendo-lockin" They apparently contribute to open ASYNC API Specification.
Elesh also surprised me when he asked his colleague Charlie (who has a sales role) to close his ears while he explained that Gravitee is open source. So we can apparently just run it ourseleves!
Wiremock Cloud - Develop in parallel and ship earlier with OpenAPI and mocking
Tom highlighted the issues of building APIs when other APIs around you are perhaps not yet available. Clearly, building apps serially (one after another) is to be avoided if we want to get things done quickly. This means we need to code to good quality mocks in order to unblock ourselves since building mocks are hugely quicker than building the real implementation.
Wiremock supports mocking with Open API (and the reverse - i.e. generating Open API from Wiremock mocks)
Tom made the point that OpenAPI doesn't describe the behaviour and only typically details happy paths without formalised scenarios.
Tom showed how Wiremock could create very functional mocks by mocking services for the hypothetical Train Travel API courtesy of bump.sh. This apparently aimed to replace the outdated Pet Store API problem.
Tom went on to show Wiremock Cloud being used to setup mocks with dynamic data and some basic operators which when combined can add dynamic behaviour.
Tom showed us a HTML page interacting with the mocked Train.
Wiremock cloud can run in kubernetes and should be able to run in a hybrid model sometime next year. WiremockCloud also supports a two way git link.
Seven key lessons on Building Great REST APIs - Vedran Cindric - Founder and CEO @Treblle
Treblle have 150k users. Vedran explained his seven key lessons on building great APIs
1. Design
- Semantics matter nouns instead of verbs. singular instead of plural
- HTTP methods each method has a specific meaning GET POST PUT DELETE
- Response codes - a shortcut to explaining what you meant to say
- Versioning - start early, start simple and always version APIS
Follow these four and it will be simpler.
2. Security
- HTTPS - completely free and easy way to kickstart your API securely
- Any Authentication is better than none
- Validate and sanitize any data you're passing through the API. The API is your brain. The holy grail is your DB. Make sure the holy grail of the DB is as clean and optimised as possible. Seen and heard a lot of people not do this.
- Cleanup your headers to avoid making it easier for others to hack you.
- Don't send proprietary info in headers (e.g. versions of applications).
3. Performance
- Caching is key to success on both the server and client side.
- Paginate data to avoid large payloads ad sizes with large data sets
- DB Performance is the number one reson for Slow APIs.
- Turn on compression to minimise response size.
4. Documentation
- Be as descriptive as possible with your endpoints and the data that needs to be sent.
- Provide request examples - because that's how most people understand things.
- Use standards for writing and reading documentation.
- Provide context for when things go wrong - Use OpenAPISpec.
5. Adoption
- Build as man SDKs as possible to make the adoption process as seamless as possible.
- Be ready to provide support on a very technical level at ever step of the process.
- Engineers expect to see beutiful developer portals.
- Simplify onboardig and make it extremely easy to get to a first request.
6. Governance
- Define a set of standards and stick to them at every step of the way.
- Implement a cetralied API versionning strategy that everyone can follow.
- Microsoft have API Governance docs that you can view.
- Etablish an API Review and approval process that doesnt suck.
- Educate people on building great REST-based APIs. Not many people are good at this.
7. Monetization
- Have a free Tier so that any engineer can easily get started and implement it.
- Provide technical and business support throughout the entire process to your clients.
- Optimise everything to speed up the time it takes your clients to onboard and make their first request.
- Build predictable pricing models that the customer can control.
The REST API Myth - Sergey Konstantinov Staff Software Engineer, Group Tech Lead - Bolt Technology (Estonia)
(Introduced by Afsha H - Capgemini. Was a Maths teacher - decided to retrain and become a software engineer).
Sergey's talk was very interesting and more of a history lesson as to how we arrived at REST becoming the default implementation choice for distributed applications.
Some key points:
- In 2000, Roy Fielding proposed REST.
- No mention of REST API or verbs in URL.
- 1981 - The birth of RPC - We write it as if it were remote or local - dont care.
- Then we went mad with RPC
- Corba, 1997 Java RMI
- None of them open. Interoperability poor.
- Then .coms happened
- Lot of users... complex to support with memory and cpu. tryig for horiontal scalability.
- HTTP on top of Corba - nightmare
- Let every node use http!! This is the idea of Rest!
- HTTP was originally build for serving text.
Why develop REST APIs in 2024?
- Can read with your eyes.
- Very popular - lots of support
- Toolkits available
- If you use REST do it for readability and convenience of usage. Well thought out URLs
- Take the best of two worlds. So specification first. Proper content-types.
Final slide - Why REST?
- Because explicit is always better than implicit.
Runtime API Governance: The Secret to Scaling Secure and Reliable APIs - Sudeep Goswami CEO - Traefik Labs
Sudeep explained that people are normally referring to design when they refer to governance. However there are more aspects to consider.
Top 5 policies for Runtime API Governance.
CI/CD
- Have CI/CD as foundation. If you do it later, then hard to achieve.
- Everything in version control.
- GitOps takes this to next level. This is orchestration. Start with it in git orchestrate in sequence
- CI/CD as the change agent.
- API Discoverability & Documentation
- Centralised catalogue
- Self-Service onboarding.
- A good API is like a good joke. If you have to explain it, it's not that good!
API Security
- Deploy an API Gateway
- Use a WAF (Web Application Firewall)
- A lot of attacks are coming from known attacks - not zero day exploits.
- Encrypt/Validate 3rd party APIs
API Change Management
- Understading the impact of API changes to minimize service failure & breaking changes.
- Change Impact analysis you need to see impact of change. But needs to be on pipeline (not sure I agree with this! If it can be included on a pipeline great, but there are other ways of achieving this)
API Obervability
- Minimum viable metrics.
- Open Telemetry - Starting to become the defacto standard.
- API Telemetry Correlation. Traefik labs can link traffic patterns to merge requests since it knows when they were deployed. Correlation is not the same as cause however.
Leveraging Domain-Driven Design (DDD) to Design Robust APIs - Tasnym Souissi - Senior IT Architect - BCG
At Tasnym's organistion, API Design wasn't considered a major issue until it was.
The Problem
- API Design was a big issue for them. They decided to introduce API driven approach to get scaleable and re-useable APIs.
- API Driven wasn't enough. We had teams in isolation not knowing about other bits of system. Stakeholders not understanding API or the technical language.
- Inconsistencies and a lack of cohesion in the system.
- Complex API Integration.
- Misalignment between business stakeholders and technical teams.
- Instead of treating this as individual bits one big system.
Solution
Build it from the core business domain area.
Split the wider business into smaller subdomains - bounded contexts
Domain model is combination of everything in your domain.
- Value objects - e.g. $5. copies -
- Entity - something with unique identity
- Aggregate - cluster of associated objects.
Key Takeaways
- Start small and focus on critical subdomain. Don't try and diagram entire company.
- APIs ae no longer just technical endpoints but true representations of your business logic.
- DDD provide the foundation.
- DDD brings teams together around a ubiquitous language.
- It can be hard to persuade people to buy-in to DDD as the benefits are more long term
Informal Chats At Booths
Magali Saul - Alliance Manager and Jérémie Veg - Co-Founder & CEO - Sopht
IBM - GraphQL demo
Using Test Containers for AsyncAPI Unit Testing and Mocking
Laurent Broudoux Co-Founder of Microcks - Director of Engineering at Postman
Laurent showed us how test containers can be used to test your service in isolation. I have used test containers myself in the past and am not keen due to:
- Requiring docker on your machine to run tests
- Slow to run
Deploying WebSockets with AsyncAPI and Architecture as Code - James Gough Distinguished Engineer & Will Osborne - Technology Associate - Morgan Stanley
My notes here are sparse. I think I had reached information overload by this point. I also remember that only one of the speakers was present and can't remember which! A few points that are perhaps worthwhile are listed below.
When building an ASYNC API:
- Bi-Directional traffic flows needs a different construct to REST.
- REST only gives you polling. You then increase polling frequency - but that sucks as it increases load (not sustainable!).
- Large payloads - impact performance
- Notifications can be achieved by webhooks but needs public URLs - These are often not practical due to restrictions on client system and considering auth/security
- WebSockets are an evolution of REST and offer ASYNC communication.
- STOMP Simple Text Oriented Message Protocol - An alternative to WebSockets
Building Cross-Exchange Trading Library with OpenAPI/AsyncAPI - Sho Kaneko Founder - Repoch
Sho had a huge amount of passion for his project. But to be honest I didn't follow it. I know it invovled OpenAPIs and ASYNC APIs. It looked interesting.
Comments
Post a Comment