Much has been written about large language models (LLMs) being a risk to user security and privacy, including the issue that, being trained with datasets whose provenance and licensing are not always clear, they can be tricked into producing bits of data that should not be divulgated. I took on reading this article as means to gain a better understanding of this area. The article completely fulfilled my expectations.

This is a review article, which is not a common format for me to follow: instead of digging deep into a given topic, including an experiment or some way of proofing the authors’ claims, a review article will contain a brief explanation and taxonomy of the issues at hand, and a large number of references covering the field. And, at 36 pages and 151 references, that’s exactly what we get.

The article is roughly split in two parts: The first three sections present the issue of security and privacy threats as seen by the authors, as well as the taxonomy within which the review will be performed, and sections 4 through 7 cover the different moments in the life cycle of a LLM model (at pre-training, during fine-tuning, when deploying systems that will interact with end-users, and when deploying LLM-based agents), detailing their relevant publications. For each of said moments, the authors first explore the nature of the relevant risks, then present relevant attacks, and finally close outlining countermeasures to said attacks.

The text is accompanied all throughout its development with tables, pipeline diagrams and attack examples that visually guide the reader. While the examples presented are sometimes a bit simplistic, they are a welcome guide and aid to follow the explanations; the explanations for each of the attack models are necessarily not very deep, and I was often left wondering I correctly understood a given topic, or wanting to dig deeper – but being this a review article, it is absolutely understandable.

The authors present an easy to read prose, and this article covers an important spot in understanding this large, important, and emerging area of LLM-related study.

I was a tiny blip in said numbers. One person, three shots. Took me three hours, but am quite happy to have been among the huge crowd.

(↑ photo credit: La Jornada, 2025.11.14)

And why am I bringing this up? Because I have long been involved in organizing DebConf, the best conference ever, naturally devoted to improving Debian GNU/Linux. And last year, our COVID reaction procedures ended up hurting people we care about. We, as organizers, are taking it seriously to shape a humane COVID handling policy that is, at the same time, responsible and respectful for people who are (reasonably!) afraid to catch the infection. No, COVID did not disappear in 2022, and its effects are not something we can turn a blind eye to.

Next year, DebConf will take place in Santa Fe, Argentina, in July. This means, it will be a Winter DebConf. And while you can catch COVID (or influenza, or just a bad cold) at any time of year, odds are a bit higher.

I know not every country still administers free COVID or influenza vaccines to anybody who requests them. And I know that any protection I might have got now will be quite weaker by July. But I feel it necessary to ask of everyone who can get it to get a shot. Most Northern Hemisphere countries will have a vaccination campaign (or at least, higher vaccine availability) before Winter.

If you plan to attend DebConf (hell… If you plan to attend any massive gathering of people travelling from all over the world to sit at a crowded auditorium) during the next year, please… Act responsibly. For yourself and for those surrounding you. Get vaccinated. It won’t absolutely save you from catching it, but it will reduce the probability. And if you do catch it, you will probably have a much milder version. And thus, you will spread it less during the first days until (and if!) you start developing symptoms.

How good can large language models (LLMs) be at generating code? This may not seem like a very novel question, as several benchmarks (for example, HumanEval and MBPP, published in 2021) existed before LLMs burst into public view and started the current artificial intelligence (AI) “inflation.” However, as the paper’s authors point out, code generation is very seldom done as an isolated function, but instead must be deployed in a coherent fashion together with the rest of the project or repository it is meant to be integrated into. Today, several benchmarks (for example, CoderEval or EvoCodeBench) measure the functional correctness of LLM-generated code via test case pass rates.

This paper brings a new proposal to the table: comparing LLM-generated repository-level evaluated code by examining the hallucinations generated. The authors begin by running the Python code generation tasks proposed in the CoderEval benchmark against six code-generating LLMs. Next, they analyze the results and build a taxonomy to describe code-based LLM hallucinations, with three types of conflicts (task requirement, factual knowledge, and project context) as first-level categories and eight subcategories within them. The authors then compare the results of each of the LLMs per the main hallucination category. Finally, they try to find the root cause for the hallucinations.

The paper is structured very clearly, not only presenting the three research questions (RQ) but also referring to them as needed to explain why and how each partial result is interpreted. RQ1 (establishing a hallucination taxonomy) is the most thoroughly explored. While RQ2 (LLM comparison) is clear, it just presents straightforward results without much analysis. RQ3 (root cause discussion) is undoubtedly interesting, but I feel it to be much more speculative and not directly related to the analysis performed.

After tackling their research questions, Zhang et al. propose a possible mitigation to counter the effect of hallucinations: enhance the LLM with retrieval-augmented generation (RAG) so it better understands task requirements, factual knowledge, and project context. The presented results show that all of the models are clearly (though modestly) improved by the proposed RAG-based mitigation.

The paper is clearly written and easy to read. It should provide its target audience with interesting insights and discussions. I would have liked more details on their RAG implementation, but I suppose that’s for a follow-up work.

As I am about to migrate it to Debian 13 (Trixie), naturally, I am set to reboot it. But before doing so:

$ w
 12:21:54 up 1048 days, 0 min,  1 user,  load average: 0.22, 0.17, 0.17
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU  WHAT
gwolf             192.168.10.3     12:21           0.00s  0.02s sshd-session: gwolf [priv]

Wow. Did I really last reboot this server on December 1 2022?

(Yes, I know this might speak bad of my security practices, as there are several kernel updates I never applied, even having installed the relevant packages. Still, it got me impressed 😉)

Debian. Rock solid.

When this book was presented as available for review, I jumped on it. After all, who doesn’t love reading a nice bit of computing history, as told by a well-known author (affectionaly known as “Uncle Bob”), one who has been immersed in computing since forever? What’s not to like there?

Reading on, the book does not disappoint. Much to the contrary, it digs into details absent in most computer history books that, being an operating systems and computer architecture geek, I absolutely enjoyed. But let me first address the book’s organization.

The book is split into four parts. Part 1, “Setting the Stage,” is a short introduction, answering the question “Who are we?” (“we” being the programmers, of course). It describes the fascination many of us felt when we realized that the computer was there to obey us, to do our bidding, and we could absolutely control it.

Part 2 talks about “the giants” of the computing world, on whose shoulders we stand. It digs in with a level of detail I have never seen before, discussing their personal lives and technical contributions (as well as the hoops they had to jump through to get their work done). Nine chapters cover these giants, ranging chronologically from Charles Babbage and Ada Lovelace to Ken Thompson, Dennis Richie, and Brian Kernighan (understandably, giants who worked together are grouped in the same chapter). This is the part with the most historically overlooked technical details. For example, what was the word size in the first computers, before even the concept of a “byte” had been brought into regular use? What was the register structure of early central processing units (CPUs), and why did it lead to requiring self-modifying code to be able to execute loops?

Then, just as Unix and C get invented, Part 3 skips to computer history as seen through the eyes of Uncle Bob. I must admit, while the change of rhythm initially startled me, it ends up working quite well. The focus is no longer on the giants of the field, but on one particular person (who casts a very long shadow). The narrative follows the author’s life: a boy with access to electronics due to his father’s line of work; a computing industry leader, in the early 2000s, with extreme programming; one of the first producers of training materials in video format–a role that today might be recognized as an influencer. This first-person narrative reaches year 2023.

But the book is not just a historical overview of the computing world, of course. Uncle Bob includes a final section with his thoughts on the future of computing. As this is a book for programmers, it is fitting to start with the changes in programming languages that we should expect to see and where such changes are likely to take place. The unavoidable topic of artificial intelligence is presented next: What is it and what does it spell for computing, and in particular for programming? Interesting (and sometimes surprising) questions follow: What does the future of hardware development look like? What is prone to be the evolution of the World Wide Web? What is the future of programming–and programmers?

At just under 500 pages, the book is a volume to be taken seriously. But space is very well used with this text. The material is easy to read, often funny and always informative. If you enjoy computer history and understanding the little details in the implementations, it might very well be the book you want.

So, there has been in effect an X exodus. This has been reported in media outlets as important as Reuters, or The Guardianresearch institutes such as Berkeley, even media that no matter how hard you push cannot be identified as the radical left Mr. Trump is so happy to blame for everything, such as Forbes…

Today I read a short note in a magazine I very much enjoy, Communications of the ACM, where SIGDOC (the ACM’s Special Interest Group on Design of Communication) is officially closing their X account. The reasoning is crystal clear. They have the mission to create and study User Experience (UX) implementations and report on it, «focused on making communication clearer and more human centered». That is no longer, for many reasons, a goal that can be furthered by the means of an X account.

(BTW, and… How many people are actually angry that Mr. Musk took the X11 old logo and made it his? I am sure it is now protected under too many layers of legalese, even though I am aware of it since at least 30 years ago…)

Today I was very pleasantly surprised, while reading the column titled «Can’t we have nice things» Kode Vicious answers to a question on why computing has nothing comparable to the beauty of ancient physics laboratories turned into museums (i.e. Faraday’s laboratory) by giving a great hat tip to a project stemmed off Debian, and where many of my good Debian friends spend a lot of their energies: Reproducible builds. KV says:

Once the proper measurement points are known, we want to constrain the system such that what it does is simple enough to understand and easy to repeat. It is quite telling that the push for software that enables reproducible builds only really took off after an embarrassing widespread security issue ended up affecting the entire Internet. That there had already been 50 years of software development before anyone thought that introducing a few constraints might be a good idea is, well, let’s just say it generates many emotions, none of them happy, fuzzy ones.

Yes, KV is a seasoned free software author. But I found it heart warming that the Reproducible Builds project is mentioned without needing to introduce it (assuming familiarity across the computing industry and academia), recognized as game-changing as we understood it would be over ten years ago when it was first announced, and enabling of beauty in computing.

Congratulations to all of you who have made this possible!

“The Comedy of Computation” is not an easy book to review. It is a much enjoyable book that analyzes several examples of how “being computational” has been approached across literary genres in the last century — how authors of stories, novels, theatrical plays and movies, focusing on comedic genres, have understood the role of the computer in defining human relations, reactions and even self-image.

Mangrum structures his work in six thematic chapters, where he presents different angles on human society: How have racial stereotypes advanced in human imagination and perception about a future where we interact with mechanical or computational partners (from mechanical tools performing jobs that were identified with racial profiles to intelligent robots that threaten to control society); the genericity of computers –and people– can be seen as generic, interchangeable characters, often fueled by the tendency people exhibit to confer anthropomorphic qualities to inanimate objects; people’s desire to be seen as “truly authentic”, regardless of what it ultimately means; romantic involvement and romance-led stories (with the computer seen as a facilitator for human-to-human romances, distractor away from them, or being itself a part of the couple); and the absurdity in antropomorphization, in comparing fundamentally different aspects such as intelligence and speed at solving mathematical operations, as well as the absurdity presented blatantly as such by several techno-utopian visions.

But presenting this as a linear set of concepts that are presented does not do justice to the book. Throughout the sections of each chapter, a different work serves as the axis — Novels and stories, Hollywood movies, Broadway plays, some covers for the Time magazine, a couple of presenting the would-be future, even a romantic comedy entirely written by “bots”. And for each of them, Benjamin Mangrum presents a very thorough analysis, drawing relations and comparing with contemporary works, but also with Shakespeare, classical Greek myths, and a very long etcætera. This book is hard to review because of the depth of work the author did: Reading it repeatedly made me look for other works, or at least longer references for them.

Still, despite being a work with such erudition, Mangrum’s text is easy and pleasant to read, without feeling heavy or written in an overly academic style. I very much enjoyed reading this book. It is certainly not a technical book about computers and society in any way; it is an exploration of human creativity and our understanding of the aspects the author has found as central to understanding the impact of computing on humankind.

However, there is one point I must mention before closing: I believe the editorial decision to present the work as a running text, with all the material conceptualized as footnotes presented as a separate, over 50 page long final chapter, detracts from the final result. Personally, I enjoy reading the footnotes because they reveal the author’s thought processes, even if they stray from the central line of thought. Even more, given my review copy was a PDF, I could not even keep said chapter open with one finger, bouncing back and forth. For all purposes, I missed out on the notes; now that I finished reading and stumbled upon that chapter, I know I missed an important part of the enjoyment.

Anyway, of course, I took the opportunity to join this experiment, and together with my Mexican colleague Tzolkin Garduño who is finishing her PhD here in France (or should I say, second to her, as she is the true leading author of our work). And here you can see the paper we submitted to the DebConf Academic Track, which will be published soon:

A retrieval-augmented-generation pipeline to help users query system-provided documentation

The corresponding source code is all available at Tzolkin’s repository in GitHub.

So, what is it about, in shorter words and layman terms?

Debian has lots of documentation, but lacks in discoverability. We targetted our venerable manpages: It is well known that manpages are relevant, well-organized documentation describing how to use each of the binaries we ship in Debian. Eric Raymond wrote in his well-known essay “The Art of Unix Programming” (2003) that the Unix cultural style is “telegraphic but complete. It does not hold you by the hand, but it usualy points in the right direction.”

Our original intent was to digest all of the information in our manpages, but we narrowed it to only the first “section” of the manual due to the limitations of the hardware a good friend lent us to play with LLMs. We took four different base, redistributable (although, yes, non-DFSG-free) Large Language Models downloaded from HuggingFace (T5-small, MiniLM-L6-v2, BGE-small-en and Multilingual-e5-small), and trained them with the 34579 pages found inside /usr/share/man/man1 of all of the existing Debian packages. We did some interesting fine-tuning (for further details, please refer to the paper itself or to our GitHub repository.

The idea is to present an interactive tool that udnerstand natural language queries, and answers with the specific manpage to which they better relate (I’d like to say “they answer best”, but Tzolkin has repeatedly tried to correct my understanding of the resulting vectorial distances).

I had prepared some images to present as interaction examples, but I’ll wrap up this post with something even better 😉 So, this is what you would get with the following queries:

• How can I submit a bug report in a package?
• I need to transcode video
• Of course, garbage-in means garbage-out. LLMs cannot tell you “I don’t have an answer for you”, so you get hallucinations (or at least, irrelevant answers): I want my lunch tickets for Brest. Guess I was hungry when working my examples…
• But I am a vegan, going on with the silliness… Anyway, do take note that, although it is not directly relatable, the reported distance is very large, so the model does tell me in a way “these are my best results, but I wouldn’t trust them too much”.
• Who is the author of the Ruby manual?

We were happy to present like this. During DebCamp, however, I was able to devote some time to translating my client into a Web-accessible system. Do note that it’s a bit brittle, and might break now and then. But you are welcome to play with it!

Play with the Web interface for our RAG for Debian manuals!

I find it worth sharing that, while we used quite a bit of GPU for the training (not too much — a couple of nights on an old-although-powerful nVidia 1070 card lent to us by the Felipe Esquivel Fund for Science and Cultural Support), all querying takes place in the CPU, and I usually get between 0.1 and 0.3 seconds per query. My server’s architecture is far from rock-solid, and it can break now and then… but at least it should respawn upon failure 😉 And it should at least be available there for a couple of weeks into August.

Anyway, this has been a very interesting journey getting my toes wet in the waters of LLM, and while current results are a bit crude, I think this can be made into an interesting tool for Debian exploration.