Speaker
Description
This talk will report on experiences with using recently developed correctness-checking idioms in prompt engineering. The idioms read as natural language statements, a property they share with "vibe coding". At least two properties distinguish the idioms from other vibe coding. First, the idioms are backed up the Fortran standard's formal grammar as defined in extended Backus-Naur form. Second, the resulting statements have unambiguous definitions in the form of defined operations from by the Julienne correctness-checking framework. As a baseline for comparison, we first used a free-form English to prompt a large language model (LLM) to generate a finite-difference approximation to a three-dimensional Laplacian operator for the Matcha T-Cell motility simulator's diffusion solver. Inspired by agile software development's test-driven development approach, we then refined our prompt by introducing Julienne tests for validation. We found that even for writing one function that junior research software engineers could likely write without difficulty, three of four LLMs prompted with precise terminology from the Fortran standard produced code with compile-time errors after two iterations. One model generated compilable code with vibe coding. Idiomatic vibe coding reduced the number of iterations required to generate compilable code but also introduced a regression: the test failure count increased.
