Note: The views expressed in the post reflect the AI capabilities at the mentioned time period. The capabilities have advanced since then.
In Q3 2023, I was working on a customer requirement for increasing the pagesize for a certain API.
Unlike a typical engineering problem which requires delving deep into design or code, this one needed deep empirical analysis of the impact of pagesize changes on the service (cpu, mem, iops utilisation) and DB (DTU) across the (thousands of) customer deployments.
Any thresholds being breached would result in scaling up of the infra resulting in additional costs across deployments.
On the other hand the restricted pagesize was hurting the customer usecase.
I used AI(ChatGPT) to generate python code snippets for:
- workloads with different pagesize
- complex OData queries
- simulate traffic patterns
- run multiple test iterations to collect data (API response time, total time for workload)
I reviewed the code snippets, tested them for smaller datasets and tweaked as needed.
As the tests ran, data was collected from our APM tool for server and DB vitals across multiple representative customer deployments across tiers.
I used AI to write more python code for statistical analysis and visualization of the collected data.
Finally I needed to map the pagesize, resource utilisation data to the cloud cost model. For this I used Google spreadsheets and took help from AI on usage of functions.
The outcome:
- visualizations for impact of different page sizes on workload processing time
- spreadsheet model for estimated cost impact of different pagesize
With this it was easy to find the sweetspot, take decisions and get executive alignment.
A few months after we had rolled out the changes, I heard from an admin in one of our biggest customers.
The workloads ran 8X faster.
No additional costs incurred as we were efficient in utilising the existing infrastructure. In some cases we reduced costs by avoiding spikes and temporary scaleups.
A win win.
It was one of the most satisfying experiences in my professional journey.
This was not a problem I could turn over to AI for various reasons.
However the specific areas I got help from AI were instrumental because of:
- faster turn around
- snippets I could review and verify before use
- areas which were not my expertise (spreadsheets) but still critical to the problem solving.
The mantra: while I don’t trust anything AI says, I can trust snippets of code or spreadsheet functions after I have reviewed and tested it myself.
I’m still the architect of the overall solution.
I understand threadbare how everything is wired together and works.
I don’t outsource my thinking or accountability. I scavenge value from what AI emits to build solutions for complex problems, do it fast, reliably and have fun in the process.
What was checked in to the repo was a battle tested custom tool to solve a complex engineering problem. Not some 5 page prompt 🙂.