Hi! My name is Eric (@16BitNarwhal). I’m an undergrad at the University of Waterloo and I recently spent a semester interning at Greptile. I’m building reporoast.coma site that uses the Greptile API and large language models (LLMs) to generate memes that roast your GitHub repo.
When I started building it, I thought it was simple – to use Greptile to analyze the codebase and maybe some kind of image model to create the meme.
This becomes more complicated for several reasons:
- It’s hard to find bugs worth roasting in the codebase.
- LLM is no joke at all.
- Image creation models struggle with meme and text formats.
Here’s an overview of what worked and didn’t work.
Create a repo-specific roast
I started directly querying Greptile’s API to analyze a code repository and generate recipes for it.
Here is my first request:
curl --request POST \
--url https:
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--header 'X-GitHub-Token: ' \
--data '{
"messages": (
{
"content": "Roast this repository in a light-hearted way"
}
),
"repositories": (
{
"remote": "github",
"branch": "main",
"repository": "16BitNarwhal/micppgrad"
}
)
}'
Disclaimer: this is it it’s really true long and there is also one disclaimer. Yeah, I don’t think it would work in a meme.
{"message":"Oh boy, let me roast this repository! 🔥\n\nAh yes, automatic differentiation in C++ - because
apparently regular differentiation wasn't complicated enough! This is what happens when math nerds decide to make
calculus even more intimidating by wrapping it in layers of C++ templates. It's like they looked at derivatives and
thought \"You know what this needs? More pointer arithmetic!\"\n\nNeural networks and automatic differentiation...
a match made in complexity heaven! It's the kind of project where even the debugger needs therapy sessions. I bet
the error messages are longer than a CVS receipt.\n\nThe fact that it has a README is like putting a \"Warning:
Hot\" label on the sun - technically helpful, but we all know we're in for pain anyway. It's probably full of
mathematical equations that look like someone fell asleep on their keyboard while holding down the Shift key
\n\nThis is the kind of repository where \"undefined behavior\" isn't a bug - it's a feature! Because who needs
predictable results when you can have the excitement of wondering whether your gradients are actually calculating
derivatives or accidentally solving world peace?\n\nBut hey, at least it's not written in Brainfuck... though
sometimes C++ template metaprogramming makes that look like a viable alternative! 😂\n\nRemember folks: If you
understand this repository on the first try, you're either lying or you're the person who wrote it (and even then,
I have my doubts)! \n\n*Disclaimer: This roast was written with love and respect for the brave souls who venture
into the depths of automatic differentiation. May your stack traces be short and your compile times reasonable!*
😉","sources":()}
To keep it short and remove any extra details that aren’t directly part of the meme, I’ve added this to the prompt:
Do not use emojis or any other special characters and do not have any introductory or concluding statements or
conversational extras.
Format your response as a single concise roast for a meme.
This seems better, it’s short and people actually have the attention span to read it:
Ah yes, another repository where developers spent more time writing documentation about how to write
documentation than actually writing code. The ultimate meta-procrastination project.
However, the output feels a bit generic. You can put it in any repo and it will still work. I want something specific that directly mentions the repository. The best way I can think of to do this is to ask Claude specifically:
Please include specific details like:
- Real file names and paths (e.g. "src/utils/hacky-workaround.js")
- Specific error patterns or anti-patterns found in the code
- Real issue numbers or PR comments
- Concrete examples of problematic implementations
Focus on concrete, specific issues rather than general observations.
Now I get different results. Sometimes it works and sometimes it fails to find relevant information in the codebase:
'The tests in test_value.cpp are more like a wish list of what should work rather than actual tests.'
and
"I apologize, but I don't see any actual code files, paths, or repository content provided in the context..."
I think it doesn’t work sometimes because I ask Greptile to do it it’s too much. Greptile “specializes” in analyzing codebases but I made these grills which may have interfered with Greptile extracting information from the codebase.
To fix this, I split the creation of roasts into 2 tasks – analysis and grilling – so that Greptile can focus only on codebase analysis.
Separation of issue discovery and generation roasting
For analysis, I prompted Greptile’s API to identify vulnerabilities in the repository. For example:
Look at many source code files and list 5 of the most egregious issues about the code itself.
Directly reference the code and files if possible.
Then list 5 issues about the repo itself. Do not list generic issues, try to list issues that
are highly specific to this repo.
The answer usually goes something like this:
Code Issues:
- In src/value.cpp, the backprop() implementation uses a recursive lambda function for topological sort which could cause stack overflow with deep computation graphs
- Neural network weight initialization in src/neural_network.cpp uses uniform random distribution (-1,1) rather than proper Xavier/He initialization, leading to potential gradient vanishing/exploding
- The Value class in src/value.cpp stores gradients as raw doubles without handling potential numerical overflow/underflow issues during backpropagation
- In src/main.cpp, the training loop lacks any early stopping criteria or learning rate scheduling, making training potentially unstable
- The MultiLayerPerceptron class in src/neural_network.cpp lacks any regularization capabilities like dropout or weight decay
Repository Issues:
- No continuous integration setup or automated testing workflow despite having a test suite in tests/test_value.cpp
- Documentation in README.md only shows usage examples but lacks mathematical explanations of the automatic differentiation implementation
- Visualization script graph.py requires manual installation of Python dependencies without proper requirements.txt
- run.sh build script lacks proper error handling for CMake configuration failures
- No versioning or release tags despite having significant functionality changes across commits
Although not yet in a “roast-y” format, it raises specific issues directly from the repo itself.
For the grill creation part, I want to eventually make meme images from it, so I prompted an LLM (Claude in this case) to grill the repo with text bubbles along with the previous ones analysis result as an input.
Here is the prompt I used
Write a single short and concise meme using issues with this repository.
Repository Context: ${analysisResults}
Include specific details like:
- Real file names and paths (e.g. "src/utils/hacky-workaround.js")
- Specific error patterns or anti-patterns found in the code
- Real issue numbers or PR comments
- Concrete examples of problematic implementations
Maintain technical accuracy while being humorous
Keep within meme format style (brief, punchy, casual tone)
No introduction, explanation, or additional commentary needed.
Output:
Me: "Uniform random (-1,1) in neural_network.cpp"
My neural net: *exploding gradients*
Time to create an image.
Create memes with image creation models
My first instinct was to use image creation models to create memes. It quickly turns out that image models struggle with text, which is necessary for memes.
Yes, it might not be the best. As far as I know, image generation models tend to struggle with text, which is necessary for memes. Another potential issue is the cost of creating an image.
So I tried to use a simpler solution: hardcoded meme templates.
I downloaded popular meme images, but there is empty text. Then, I hardcoded the rectangular positions of each image to place the text.
For example:
bubbleRects: (
createRect(648, 1176, 24, 576),
createRect(648, 1176, 624, 1176)
)
Now for the text to actually go with these images, I need to repeat the LLM that produces the roasts. For each image template, I have a different “meme image description” added to the prompt. example
The image has 2 rows with Drake rejecting the first text bubble and Drake happyly accepting the second text bubble. Format your response as a JSON array of strings. Make sure the array is has length 2, representing the number of
text bubbles.
I parse this list and use canvas node to insert text directly into this template images.
Another problem: LLM doesn’t always give me a list and it doesn’t always match the length! Sometimes I get something that doesn’t parse like “Here is your list: (A, B)” or a list that uses the wrong size like “(A, B, C, D)” . So, I created a retry policy where every time it failed, I reprompted the LLM on its previous response and added “This is not a JSON formatted array. Try again” or “The array must be size 2 .Try again”.
Anyway, here is the output:
It doesn’t look good. Of all. That means the text is there at least. I should probably add the phrase:
And make the text as big as possible, I forced all the font sizes until the text was too big and overflowing.
This is a bit on the hackier side, but it works and keeps the text consistent:
While there were some bumps and hacky implementations along the way, I had a ton of fun building an app designed solely for grilling projects.
Here are some finished outputs:
You create your own memes at reporoast.com and let me know what you think!
https://greptile.com/reporoast/reporoast-opengraph-2.png
2025-01-06 17:07:00
