A comparison between 3 command line interface AI coding tools completing a task to implement an iOS networking client using an OpenAPI backend API specification. The tools compared are; Aider-chat, Claude Code and Codex CLI with each tool given the same prompt and project starting point. The coding task was not especially challenging, so the point of the comparison is not to rank LLMs using benchmarks (that’s already done extensively), but to compare the products themselves, their cost and ability to operate in completing a simple example in the domain of mobile app development.
| Tool | Time to Implement | Cost | Initial Prompts | Compiler Fixes | LLM |
|---|---|---|---|---|---|
| Aider | 5 mins | $0.05 | 2 prompts; 71 and 85 words. | 6 ‘fix’ prompts to fix build errors | Main model: o3-mini with diff edit format Weak model: gpt-4o-mini |
| Claude Code | 1-2 mins | $2.16 | 2 prompts; 71 and 85 words | none | claude-sonnet-4-20250514 |
| Codex CLE | 15-20 mins | $0.94 | 4 prompts; 22, 43, 34 and 51 words. | 3 ‘fix’ prompts to fix build errors | codex-mini-latest |
In previous articles, eg Generative AI App Builder - Introduction, I built a system of scripts to essentially do 2 things:
A lot has changed since this article was written and much of the work needed previously is now available out of the box in the tools.
Each tool was asked to build an API Endpoint Specification in swift using the following OpenAPI JSON specification:
Using the following implementations as an example:
What is it: Aider-chat is an open source and free command line tool allowing you to write and edit code using natural language prompts, inferencing with many of the popular LLMs available. It maps the structure of your repo, makes the requests to the LLM, and edits your source code providing a summary of the changes through the terminal.
1st Prompt:
Following the example of ExampleMoviesAPI, using http://localhost:8001/openapi.json for endpoint details add new cases to CafeProductsAPI. /products will map to Array of DecodableCafeProduct and /products/{product_id} -> GET will map to DecodableCafeProduct and /products/{product_id} PUT will map to DecodableCafeProduct. For each endpoint, all responses should be mapped to a decodable response struct separate to the domain model like ExamplePopularMoviesResponse Only decode the values required by the domain model.2nd Prompt:
Following the example of NetworkMoviesAPIClient, using CafeProductsAPI for endpoint details add new methods to CafeProductsClient. /products will be decoded to Array of DecodableCafeProduct then the Result success block will map to Array of CafeProduct. /products/{product_id} -> GET will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. /products/{product_id} PUT will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. This is a first draft, keep the solution simple. IMPORTANT: implement the solution without asking any questionsCode Produced:
Total Cost: $0.13 (whole session)
What is it: Claude Code is an agentic coding tool from Anthropic that runs in your terminal, understands your codebase, and uses natural language commands to implement code changes.
1st Prompt:
Following the example of ExampleMoviesAPI in /AIGenerator/Sources/Package_NetworkingAsyncEndpointDescriptor/NetworkingExample/NetworkingExample/ExampleMoviesAPI.swift, using http://localhost:8001/openapi.json for endpoint details add new cases to CafeProductsAPI in /AppGenAISwiftUIStarter/AppGenAISwiftUIStarter/Repositories/CafeProductsAPI.swift. /products will map to Array of DecodableCafeProduct and /products/{product_id} -> GET will map to DecodableCafeProduct and /products/{product_id} PUT will map to DecodableCafeProduct. For each endpoint, all responses should be mapped to a decodable response struct separate to the domain model like ExamplePopularMoviesResponse Only decode the values required by the domain model.2nd Prompt:
Following the example of NetworkMoviesAPIClient, using CafeProductsAPI for endpoint details add new methods to CafeProductsClient. /products will be decoded to Array of DecodableCafeProduct then the Result success block will map to Array of CafeProduct. /products/{product_id} -> GET will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. /products/{product_id} PUT will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. This is a first draft, keep the solution simple. IMPORTANT: implement the solution without asking any questionsExample Claude output:
Example Output:
⏺ I'll implement the CafeProductsClient methods based on the example of NetworkMoviesAPIClient. Let me first examine the structure and create a todo list.
⏺ Update Todos
⎿ ☐ Examine NetworkMoviesAPIClient for reference
☐ Examine CafeProductsAPI for endpoint details
☐ Implement GET /products method in CafeProductsClient
☐ Implement GET /products/{product_id} method in CafeProductsClient
☐ Implement PUT /products/{product_id} method in CafeProductsClient
⏺ Update Todos
⎿ ☒ Examine NetworkMoviesAPIClient for reference
☒ Examine CafeProductsAPI for endpoint details
☒ Create CafeProductsClient class and protocol
☒ Implement GET /products method in CafeProductsClient
☒ Implement GET /products/{product_id} method in CafeProductsClient
☒ Implement PUT /products/{product_id} method in CafeProductsClientCode Produced:
Total Cost: $2.16 (whole session)
What is it: Open AI’s Codex CLI is an open-source local coding agent that runs through your command line interface. As advertised you can “pair with it to write, edit, and understand code”. I should be clear that as stated on the Codex CLI github repo, “Codex CLI is an experimental project under active development. It is not yet stable, may contain bugs, incomplete features, or undergo breaking changes.”
Following the example of ExampleMoviesAPI in /AIGenerator/Sources/Package_NetworkingAsyncEndpointDescriptor/NetworkingExample/NetworkingExample/ExampleMoviesAPI.swift, using http://localhost:8001/openapi.json for endpoint details add new cases to CafeProductsAPI in /AppGenAISwiftUIStarter/AppGenAISwiftUIStarter/Repositories/CafeProductsAPI.swift. /products will map to Array of DecodableCafeProduct and /products/{product_id} -> GET will map to DecodableCafeProduct and /products/{product_id} PUT will map to DecodableCafeProduct. For each endpoint, all responses should be mapped to a decodable response struct separate to the domain model like ExamplePopularMoviesResponse Only decode the values required by the domain model.Result: We had a lot of…
278s Thinking...
310s Thinking... system
⚠️ Network error while contacting OpenAI. Please check your connection and try again.To make this work I split the prompt:
Single prompt used by other tools split for open-ai codex:
Following the example of NetworkMoviesAPIClient, using CafeProductsAPI for endpoint details add new methods to CafeProductsClient. /products will be decoded to Array of DecodableCafeProduct then the Result success block will map to Array of CafeProduct./products/{product_id} -> GET will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. /products/{product_id} PUT will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. This is a first draft, keep the solution simple. IMPORTANT: implement the solution without asking any questionsAfter the split followed by a lot more:
Thinking...Output from open-ai codex
I've added the three endpoint cases to CafeProductsAPI, conformed it to the Endpoint protocol, and implemented its required properties (path, method, urlComponents, headers, httpBody) following the pattern
in ExampleMoviesAPI. Specifically:
* **Conform** `CafeProductsAPI` to `Endpoint`.
* **Added** cases:
* `.getProducts` (GET / products)
* `.getProduct(productID:)` (GET / products/{id})
* `.updateProduct(product:)` (PUT / products/{id})
* **Implemented**:
* `path` switches to the proper "products" paths.
* `method` returns `.get` for the GET cases and `.put` for the update.
* `urlComponents` is empty for all cases.
* `headers` includes `Content-Type: application/json` for the PUT case.
* `httpBody` JSON-encodes the `CafeProduct` for PUT, and is `nil` for GETs.1st result: failed to build:
Instance method 'encode' requires that 'CafeProduct' conform to 'Encodable'Even after prompting with the 2nd half of the requirements:
/products will map to Array of DecodableCafeProduct and /products/{product_id} -> GET will map to DecodableCafeProduct and /products/{product_id} PUT will map to DecodableCafeProduct. For each endpoint, all responses should be mapped to a decodable response struct separate to the domain model like ExamplePopularMoviesResponse Only decode the values required by the domain model.Open AI did then create a new DecodableCafeProduct that could be encoded to JSON, but didn’t include it in the enum as the associated value.
/products will map to Array of DecodableCafeProduct and /products/{product_id} -> GET will map to DecodableCafeProduct and /products/{product_id} PUT will map to DecodableCafeProduct. For each endpoint, all responses should be mapped to a decodable response struct separate to the domain model like ExamplePopularMoviesResponseCreating the network client (as opposed to the API), was also split across two requests:
1st Prompt:
Following the example of NetworkMoviesAPIClient, using CafeProductsAPI for endpoint details add new methods to CafeProductsClient. /products will be decoded to Array of DecodableCafeProduct then the Result success block will map to Array of CafeProduct. 2nd Prompt:
/products/{product_id} -> GET will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct. /products/{product_id} PUT will be decoded to DecodableCafeProduct then the Result success block will map to CafeProduct.Codex-CLI got there in the end, but out of all the tools it struggled the most with the test prompts. At this time it isworth considering the the warning from Open AI ”… is an experimental project under active development”.
Total Cost: $0.94 (whole session)