blog dds: 2026-07-08 — Why agentic AI needs better experts

Over the past few days I changed the way the uutils project’s sed program handles data to default from characters to raw bytes. This improves compatibility with GNU sed and also performance. Given the change’s size and extent (13 changed files, 1740 insertions, 609 deletions), I worked with an AI agent (OpenAI Codex), which allowed me to experience first-hand both its power and limitations.

On the one hand, Codex guided me correctly toward the task and handled it successfully both through the initial prompt and through refinements. It handled expertly a lot of tedious grunt work, saving me time and effort. At all stages the modified code compiled and run successfully. (Having a large set of unit and integration tests as well as CI checks helped guide the agent.)

On the other hand, I realized that to obtain production-quality code, Codex’s work needed careful reviewing and extensive expert guidance. At the end of this post I have included all prompts that I issued, with the ones that guided Codex to improve the code formatted in bold. From the 78 prompts and sub-prompts 61 (78%) asked for improvements.

So,

if you’re learning to program think about how you’ll obtain the knowledge and experience required to recognize such issues and prompt an agent to fix them;
if you’re managing software development or HR, plan how you’ll onboard new developers to obtain these skills; and
if you’ve decided to outsource coding completely to AI agents, consider whether you’re OK with your code gradually deteriorating and acquiring technical debt by missing improvements such as the ones I identified and asked Codex to fix.

To be fair, agents continuously improve, and many of the prompts I gave, such as those involving comments and unit tests, could have easily be handled by a more capable agent or even with a suitable initial agent configuration. But, as coding agents improve, the tasks we give them will also become more demanding, as will become the required reviewing and expert guidance.

Finally, there’s AI’s monetary and environment cost, which may mean that not all work that can be done by an AI agent is worth doing it thus. (I made several changes on my own, mainly to save time and waste.) At the end of the session, Codex (using gpt-5.5) reported Token usage: total=4,298,673 input=4,113,242 (+ 110,110,848 cached) output=185,431 (reasoning 30,920).

ChatGPT calculates that with current prices the session’s tokens would cost about $80. (I used Codex through my OpenAI subscription and current token costs are probably subsidized, but the figure gives the true cost’s likely magnitude.) My guess is that for sustained use the cost would mirror that of a developer salary. This is not unreasonable given the benefit I received, but organizations certainly need to weigh that cost into their budgets.

I was surprised by the session’s large environmental impact. ChatGPT (based on Mistral’s Le Chat reporting) gave me ~326 kg CO₂e and ~12,871 L of water usage. To put these values into perspective the CO₂ emissions are similar to a 2,5000 km trip in an efficient diesel/gasoline car or one short/medium-haul passenger round trip. And if we price these externalities, the CO₂ emissions cost €26 through the EU ETS carbon price or $42–117 through the US EPA social cost of carbon. The cost of the water used in the local Athens domestic water tariff would be ~€4.50–41. Again, these costs are not astronomical, but also far from negligible.

Session Prompts

Below are the prompts I gave during the session. Those associated with improvements I spotted are formatted in bold. Note that some of the unit tests I asked were missed in previous versions, so they’re code improvements requiring human prompting but not the agent’s fault.

How can I see GNU sed compatibility score?
How can I update the chart?
What is the highest priority change to reduce the number of failing GNU compatibility tests?
Good. So modify the code to Represent pattern space, hold space, transliteration input/output, and script literals as bytes internally, converting to UTF-8 only for Unicode-aware regex paths that actually require it. Ensure that all local tests continue to pass. Also GNU tests should improve. Be careful not to break the fast_ optimizations.
change the struct Transliteration field named fast into byte_fast.
Isn’t there a library function for script_text_to_bytes?
Add the comment and unit tests.
I think script_text_to_bytes is the wrong approach. Handle the script as bytes.
In the as_str()? cases shouldn’t we report an appropriate input_runtime error where applicable?
Should the ScriptCharProvider::new initializations simply deal with bytes?
I run git subst current_bytes current ; git subst next_line_bytes next_line to make the code mirror the original. But I broke it. Please fix it.
If we have current_char shouldn’t current be current_bytes?
Shouldn’t provider.rs deal with bytes (rather than “logical characters”)? Byte sequences should be converted into characters / strings only when required, e.g. for FancyRegex and transliterate when processing UTF-8 input. I added character_mode in the context to help you with that.
Arrange for parse_transliteration to return a variant based on context.character_mode, so bytes or string. Then parsed_script_text_to_utf8 wouldn’t be needed. Also shouldn’t the regex pattern start life as bytes and only be upgraded to string for FancyRegex? Also, shouldn’t LiteralMatcher operate on bytes?
Why did you remove the NEEDS_RE-based logic?
Why not use the ByteRegex matcher?
Good, do that. You can see the original implementation through Git HEAD. Also when adding new functions always prefix them with a documentation comment.
Regex::new should get passed context.character_mode and only create a string from UTF-8 (if needed) when mode is Utf8. In Byte mode Regex::new should fail with an error (back-references are not supported in byte mode) if it needs to construct a FancyRegex matcher. Also, add integration tests to test s and y command behavior under diverse input / scrips / LC_ALL settings.
Here are some suggested improvements. If you disagree let’s discuss before going to work.
- Change the Transliteration lookup fn into lookup_char (to contrast with lookup_byte).
- I think push_script_char should follow the bytes.push only in Bytes mode, otherwise it can create invalid Unicode scripts. In Unicode mode it should push the character’s corresponding Unicode representation.
- To minimize churn let’s have line.current() return a char.
- Rather than using extend_from_slice(line.current_bytes()) consider using the more readable push(line.current_byte()).
- Define an ERR_ constant for “transliteration strings are not the same length”
- Implement (and export) push_script_char only in delimited parser.
- Add unit tests for newly introduced functions.
- Move ParsedTransliteration into command.rs.
Shouldn’t self.utf8_verified.set(true); be in set_to_string? (I’ll fix it).
So I’ll just move the assignment, right?
Why are you encoding Regex Bytes >0x7f as \x? Can’t regex bytes handle them as is? Just tell me.
Shouldn’t Match::from_bytes avoid constructing text in Bytes contexts?
Why did you change the implementation of fn retreat?
Doesn’t new_ucmd() provide a way to specify the environment?
A few more fixes.
- No need to pass character_mode to byte_regex_pattern. Just examine character_mode when building the regex.
- Avoid having Match::from_bytes constructing text in Bytes contexts.
- Add unit tests for byte_regex_pattern
- In let cmd_str = pattern Don’t convert into UTF-8. Rather adjust and continue the flow with bytes using OsStr/OsString.
- The same when opening files for the r and w commands.
- fn list should also work correctly in byte mode without converting into a String. Be careful to avoid duplication in the way characters / bytes are shown.
- Revert the unneeded change of fn retreat()
- For the tests requiring an environment use new_ucmd!().env(...) rather than the custom run_sed_with_locale. In general follow the logic of existing tests.
Almost there!
- In Regex::new issue a USimpleError rather than construct one from scratch
- In non-Unix os_string_from_bytes try from_utf8 and issue an error on failure.
- Use a single implementation of os_string_from_bytes (put it in delimited_parser writer) rather than duplicating it.
- Add unit tests for readable_char readable_ascii_byte.
Also add unit test for write_list_item.
write_list_item calls len for each new item, making its complexity N². Encapsulate the function and the length in a class to avoid this.
Any other GNU failures that can be addressed through these changes?
Have a look at 8 bit.
What do you mean by literal raw-byte pattern?
Show me the specific change
There’s no need to test for both NEEDS_RE.is_match and NEEDS_FANCY_RE.is_match. It’s enough to test only for the first, right?
No need then for a function encapsulating the test.
What happens if no locale environment variable is set?
Fix the clippy errors.
Suggest next juicy target for improving GNU compatibility.
Remove Transliteration::is_byte_safe.
- Add unit tests for os_string_from_bytes, parsed_bytes_to_utf8, parse_character_class with '[', parse_transliteration_for_mode, Transliteration::apply_match, Translitaration::from_bytes, lookup_byte, write_line_bytes, the regex::bytes::RegexBuilder::new path, Regex::captures_iter, and Regex::find.
Isn’t it more efficient to have the is_byte_identity field? If so, just loose the trivial getter.
why pub(crate)?
How are the other fields accessed?
Add unit test for invalid characters file path
Add unit test
- for unterminated substitute replacement
- for \q in compile_replacement
- for ~step in an RE address
- for extra characters at the end of a command in parse_command_ending
- EOF in compile_replacement
- Unescaped newline in compile_replacement
- Unterminated replacement in compile_replacement
- Modifiers in empty RE in compile_subst_command
- compile_trans_command
- [ at EOL in parse_character_class
- Character class [:] in parse_character_class.
- Backslash character in parse_transliteration_bytes
Create a file prompts.md with the prompts I issued in this session. Format code identifiers in backticks. Format prompts that ask for corrections to previous work in bold.