J
As a remote non-US employee of a US company Thanksgiving for me is side-project day. Today I'm going to be hacking on an LLM plugin for Sphinx. I'm most interested in generating summaries of content for cross-referencing during the documentation build step. I'm going to live-post here as I go!
19,321,360 views
J
Replying to
@jacobtomlinson
This project was hugely inspired by Aarti Jha's talk at PyData NYC "The Art of Compression: Crafting Insightful Summaries with LLMs" www.youtube.com/watch?v=kCCC...
5,200 views
J
Replying to
@jacobtomlinson
I've started a new repo and used `hatch init` to create the boilerplate structure. I'll push code there frequently today. github.com/jacobtomlins...
5,200 views
J
Replying to
@jacobtomlinson
I've created some sample documentation with sphinx-quickstart and then I'm using sphinx-autobuild to rebuild the docs every time I make a change. I'm also using uv to keep dependency management clean. www.sphinx-doc.org/en/master/us... pypi.org/project/sphi... github.com/astral-sh/uv
5,200 views
J
Replying to
@jacobtomlinson
I'm going to need some boilerplate documentation to start summarising. This feels like a good use case for Copilot to hallucinate some entertaining nonsense.
5,200 views
J
Replying to
@jacobtomlinson
I have a very basic sphinx extension written out now. I can add a custom admonition called docref which will link to another page and include a summary. Right now the summary is hand written and it doesn't link though.
5,200 views
J
Replying to
@jacobtomlinson
Ideally I want to be able to attach a debugger during the build. I use VS Code so let's hook up the sphinx build to the debugger. Calling `sphinx-autobuild` is functionally the same as `python -m sphinx_autobuild` so we can use the Python Module debugger to invoke it.
5,200 views
J
J
Replying to
@jacobtomlinson
I really appreciate Copilot for Qs like "I have the name of the Sphinx document, how do I load that document as a string?" Past me would spend an hour digging through the Sphinx API reference to figure this out. `self.state.document.settings.env.app.builder.env.get_doctree(doc_name).astext()`
5,200 views
J
Replying to
@jacobtomlinson
I have everything in place to load the contents of the page and call a function to generate the summary. Then replace the contents of the directive with the generated summary. Now I need to wire up the LLM to do the actual generation. I also need to think about caching to avoid repeat LLM calls.
5,200 views
J
Replying to
@jacobtomlinson
First summary is looking good! I'm running ollama on my PC with an old GeForce GTX 1080 GPU. Then using langchain to invoke the llama3:8b model.
5,200 views
J
Replying to
@jacobtomlinson
Ollama is super nice for playing around with LLMs locally. github.com/ollama/ollama
5,200 views
Replies
J
Replying to
@jacobtomlinson
My new favourite LLM generated sentence: "Avoid telling the pig that you have more apples if you do not have any left, as this can lead to disappointment and frustration."
5,200 views