frankie-tales

The problem is not only if, but how.

2026-06-01T13:30:00Z

Ok, such dramas and the whole topic are starting to be annoying. The last episode was the rsync project with the well-respected Andrew Tridgell as maintainer, who initiated a Claude-based development effort for the 3.4 series a few months ago. The rsync tool is a well-known, widely used tool for copying files between hosts in a smart way, includes a daemon to optimize file transfers, and serves as an entry point for synchronization.

Only at the end of May, when some regressions began to appear, this step by the tool's main developer triggered the opening of a rather harsh issue regarding the use of AI agents in development. One of the most consistent changes in the source tree has been the introduction of Python helpers to replace the tool's test coverage. This is not a new type of controversy; it appears as a quite common pattern. AI tooling for programming (or automatic programming, as Antirez calls it) based on specifications and natural language appears to be quite widespread nowadays.

It is no novelty that such applications provoke a divisive reaction, with fans and detractors of a tool that, at first glance, appears almost magical for creating new programs from scratch in any language, or for reviewing existing code, or for evolving any existing project through extended, progressive pull requests. This is particularly annoying for many contributors when the AI agent is blindly used as a committer or to create PRs or issues. The whole controversy comes into dramatic focus when the project is a prime-time one, as in the case of rsync.

Note that rsync has been with evidence in quite low-intensity maintenance in recent years, with several hundred issues open and almost 50 PRs proposed but still pending. This is quite normal for seasoned FOSS projects, like it or not, and specifically for low-voltage tools that have most of their life-cycle in the past. Rsync is a tiny utility based on less than 150,000 lines of C code, started about 30 years ago by Tridge. In 2024, he stepped up again as a maintainer, likely due to the project's overall status (merely not abandonware, but almost such). This is, unfortunately, the known problem of too many FOSS projects to count: they are old, seasoned programs developed years ago and substantially frozen, but for minimal maintenance. They are often written in legacy languages, such as pure C, which are quite far from the interests of average Gen Z developers. Development teams are minimally set up, even under a foundation umbrella, and interest in advancing them concretely is equally minimal. They are what I call ‘good enough’ projects, often of widespread use. They could be so legacy that they are considered carved in stone.

Of course, in 2026, the temptation to revitalize such legacy programs with AI tooling is extremely high. It is higher when developers are overwhelmed with many other tasks, and you know, maintaining such a project can be extremely boring. But when the tool in question is considered a brick of the whole FOSS basement and used as such by tons of reverse-dependent projects, the whole operation can be extremely prone to controversy and intolerance. There are not only problems of quality assurance for the final result, but also ethical and philosophical questions about the use of AI tooling. All such aspects cannot be ignored, and some kind of caution is required.

To introduce AI tooling into such a project, the first step is to reach an agreement among all parties involved. That would lead to an accepted, explicitly stated AI policy (which apparently lacked in the case of rsync). The most accepted approach currently is a rigorous human-in-the-loop development policy (as accepted in the Linux kernel, for instance). Without them, any forced use of AI tooling for community-based FOSS projects is doomed to failure and controversy, which could ultimately lead to a fork and the dispersion of effort and energy. This consideration implies that the project has already adopted some form of governance. Once the IF phase is passed, the HOW step must be consequent. It is totally unacceptable that a human-in-the-loop becomes a purely formal step before committing. In other words, the project committers need to strictly review and commit to avoid regressions and subtle bugs, which could easily be present with or without the AI contribution, but for sure are a risk when one touches legacy code. The human committer is responsible. Being proactive is a duty and cannot be neglected. Better to do nothing than mess up working code. I don’t know if this has been the Tridge’s case, but regressions have happened, which isn’t a good signal, and the whole thing could lead to unexpected upset.

But, but, but... Wasn't the AI tool the magic silver bullet to solve all software issues and productivity? Well, dudes, I’m afraid that you need to change your mind. There’s no silver bullet, even 30 years later. There’s no magic recipe to solve all problems with undersized projects, and someone has to pay the debt.

We still need engineers who do their job, damn it.

The Web is dying, long live the World Wide Web.

2026-05-27T13:30:00Z

I already dealt with the status of the WWW in the recent past. See, specifically, the authoritative opinion of Sir Tim Berners-Lee on that. Multiple dynamics have upset the initial idea of a distributed, democratic, and influential worldwide system, as conceived in the 90s, including the roles of BigCorps, socials, and geopolitics. What appears to be the last brick in the grave of the WWW is the current AI wave, which introduces two distinct elements of breaking an already compromised equilibrium.

It is now evident that a large part of the WWW content is AI-generated [3,4], often with very low quality and care. Let me point out, for instance, articles in the well-known Medium.com magazine (but that’s not specifically a problem of it, it is a general one): it is not a novelty that quite a lot of authors there are intensively writing most of their content via AI, with just minor human contributions. This is not new: in the past, the figure of the ghostwriter (who wrote slop contents by weight) was quite widespread and even present in prime-time magazines, but now AI tooling has exacerbated this approach. This is what is generally considered the enshittification of the web, and it started even before the AI hype, with content generated by extensive copy-and-paste and the addition of announcements, clickbait titles, sponsored articles, etc.

At the same time, the web search systems are also undergoing significant changes. Google introduced AI overviews, but people also changed their approach to searching for information, and that was not dependent on Google's or any other search engine's suggested overview. The possibility of having a structured summary of any argument, instead of a long list of well (or not so well) centered references, changed many people's views of the whole information search topic. Unfortunately, this also deeply impacts the overall sense of sponsoring a search engine or maintaining a site of content about any topic. For instance, why maintain a whole site about tourism in Berlin if the same information can be obtained in a short form via AI independently, and no one will visit such a site? And above all, how will content creators continue to monetize effectively in such conditions [1]?

If, as seems likely after Google I/O 2026, the next search engines will be simply AI overviews, mainly with sponsored results (typically by secretely steering AI summaries), what about independent and non-profit content? To be clear, already most traffic (I mean >90%) on small websites is bot-generated. Of course, that includes AI bots and general crawlers. The duration of such visits is typically under a few seconds. Of course, such a status is not a big problem for non-profit sites, but it can cause some frustration. That was true also before the autumn of 2022, indeed. What will be the final result of such a process is quite evident to me: the enshittificated, mechanical production of web content will go away because it will become only noise on the net, without any real revenues (or value); as it has started, it will surely go mostly to a dead end. Of course, until it can still capture a fraction of users' attention, it will stay here, but in my opinion, it will only be a matter of time before such low-profile content ends. What will remain is probably paywalled or institutional pages (only those with a decent value per se), non-profit/community websites, and AI-protected sites (with a continuous risk of abuse by new generations of autobots). Free access with announcements will become a thing of the past, as the number of active human readers declines. Mass production and poetry, to summarize.

Surely, easy monetization and SEO will soon be a keepsake. It has also been visible for ages in alternative media (such as video-based content), which is currently much less accessible to AI generation solutions. Even in such a domain, content creation has become more selective, and valuable assets are quite distinct from a more general, low-quality production. My idea is that, in the long run, most content creation (in any format, text, audio, or video) will follow a bimodal distribution (with low and high quality peaks) around its general quality, and the best quality will generally require some form of subscription, because of the involved efforts for production. Indeed, I would expect some concentration among a relatively few actors for any topics. Quality is neither spread nor democratic. One will need to exit the mass of low-quality profiles, which are and will be the majority of what is available out there on the web.

That’s already visible in the traditional mainstream media: most of their web-based content is pure garbage, and that was true even a few years ago; now it's paroxysmal. I found years ago that most of the so-called news on mobile devices was largely disseminated through announcements, dialogs, and sponsored content, making it completely unusable and unfocused, for me. Nevertheless, the mobile-first approach has already been the dominant UX for the average user for ages. Even the AI-generated content could largely become targeted to AI bots instead of humans, with unclear implications about readability and form. But the average user gets used to the most incomprehensible things, so who knows?

Therefore, the concrete perspective (or the crude reality) is a web full of AI-generated content or walled gardens, with few non-profit corners that publish mainly for fun, along with a few authoritative sources (such as Wikipedia [2], a knowledge source that will need to be defended and encouraged as well), which are becoming increasingly difficult to discover in the old way by humans.

We all become like modern Diogenes walking with a lantern on the net, during the daylight, looking for the true web value with mixed luck...

A giant step backward at a time when there were manually maintained indexes of interesting sites organized by topics. Damn it, sounds familiar. Not so bad, wasn’t it? That’s the core of the Fediverse, basically: some micro-universes oriented to small communities of focused users, all navigating in a global ocean of mere s*t.

References

The only good AI is a dead one: myth and reality of AI tooling for code

2026-05-05T13:00:00Z

Some months ago, I participated in this Mastodon poll. My thesis was that, in general, there is no presumptive quality flag that can be added to a product, whether it was created with AI-aided tools or not. My opinion sparked a small, heated controversy among some readers.

Labeling a product as AI-generated and expecting low-quality results is simplistic and prejudiced. One expects that the resulting product is accepted without review or handcrafted changes, after multiple adjustments to the LLM prompt through an iterative process. This is clearly false in general for coding (vibe coding, per se, is not a black-and-white picture; it is full of shades of grey), but it can also be applied to other creations.

Even the use of AI tooling has hundreds of nuances, especially in many articulated processes of creative production across multiple fields. AI tools or not, the human contribution is still central in the process. One can abdicate to such a role, or be fully responsible for the final result, with accurate changes and handcrafted work based on a rough draft generated by AI. The presumption of labeling AI helper use as a signal of low-quality production negates the importance of the human-in-the-loop. The point is that quality should be considered an objective aspect of any product, with or without human authoring: having a handcrafted product is not necessarily a symptom of quality per se, and the same goes for the opposite if the product used AI tooling at any phase of its creation. At least for programming, we had spaghetti code for ages, well before AI agents.

If you take a walk around GitHub and look at too many projects to enumerate, even without any AI intervention, you will find a lot of half-finished, incomplete, alpha-quality, obsolete, or partially working code that would need a good number of deep refactorings to be considered for production use. That’s not a problem with AI use; it’s simply due to the not-too-recent shift in FOSS coding as the mainstream approach to writing programs. Opening a GitHub portfolio assumed almost the same importance as opening a LinkedIn profile for techies. In many cases, such proof-of-concept products have been sitting on a developer’s shelf for years. Today, they populate their GitHub (or any other hub) repos, instead.

On the opposite side, one could read a recent Antirez’s post, which shows how much a meticulous human-in-the-loop approach for automatic programming can be productive and reasonable. Not secondary; such an approach should be currently considered essential to fully admit a copyrightable contribution to existing or new code, as explained in Simone Aliprandi’s recent book.

In conclusion, pretending that AI tooling produces only slop is clearly prejudicial, not different from past anti-FOSS prejudice that unpaid work done in free time cannot be of good quality. Does it sound familiar? Sure, there are a lot of low-quality projects out there, but again, that’s not necessarily the case, and there are also many very good FOSS projects, created and maintained with great care. Again, a license type and development workflow are not enough metrics to judge software quality, nor is the use of AI sufficient for the same purpose. And that’s true for coding as for other creative works. That’s not that simplistic, dudes.

Guess what? 97% of 3,456 respondents in the Mastodon poll answered that an AI mark on a product is a good idea for a presumptively low-quality design. I’m quite sure most of them are also AI users in some form, of course, and that says a lot about the future of this AI era and human ingenuity.

The Artificial Author: copyright and copyleft in the AI era

2026-04-30T11:00:00Z

I recently finished reading Simone Aliprandi’s second edition of a book I would recommend to many people interested in either authoring software and databases or making creative works. ‘The Artificial Author’, as its title suggests, deals with the role of generative AI in creative jobs seen from a legal point of view. This massive book seeks to shed light on a central yet complex issue that, in recent years, has affected many people, not only in the IT sector but also across many sectors of the creative industry.

As often happened in the past, technological changes largely anticipated society's ability to adapt to new developments, particularly in high-impact areas such as copyright, intellectual property, and privacy. Frankly, it is impossible to cover in this review all the aspects of Aliprandi's work that it touches on and explains. Of course, the book is purely informative, but it summarizes where we are currently, from the perspective of legal implications for code and other content generated by AI tooling. The topic is an open problem for several reasons, including the undeniable fact that copyright legislation and the legal systems vary from country to country. So, for instance, what can apply in the US needs to be reconsidered for Europe. Actually, a series of legal trials is still ongoing since the start of the main new AI wave in the autumn of 2022, and some of them have not been finalized or reached an out-of-court (and so off-record) settlement.

That’s significant in common-law systems like the US's, while in Europe, the whole process is much slower and settles based on principles and legislation, not specific trials. The questionable points are:

Is the whole training process for deep learning networks indeed legitimate under the fair use right? In most cases, it seems so, given the current state of things.
Who owns the AI-assisted outputs, and are they copyrightable? The safest answer is: it depends.
Should/could the AI outputs be considered derivative work? Moreover, if legally possible, should it be considered legitimate under any conditions?
If the outputs of independent prompts and refinement are somewhat similar, is there any condition of plagiarism?
Is there really any hope of a clean-room implementation, or even a fully original one, with AI tooling?

While it is clear that a fully AI-created work is not copyrightable under US law, the legal implications of a human-in-the-loop approach remain nuanced at this time. For sure, it seems that a simple prompt is not enough to claim rights on anything, and the whole process should be qualified by a significant human contribution. In the US, the meaning of copyrightable creative works implies involvement of skills, labor, and judgment by a person, and a simple prompt is simply not enough in most cases.

That also explains why, in some contexts, AI tooling should be considered with a grain of salt, and, for sure, the use of any AI-assisted contribution to co-authored/commissioned work should be clearly announced and stated as admitted: it could have future (if not current) impacts on authors' copyright claims.

Specific book sections also address recent European regulatory actions, as well as a general overview of pending and interesting litigations in the field in the European context. Indeed, the EU AI Act deals mostly with privacy.

I strongly suggest that interested people have a deep look at Simone’s book, because I have read too many naive claims here and there on the net about this matter. Of course, what is legally admitted could also be illegitimate or unethical, but prospects and past experiences (even considered in the book) are about future trade-offs and proper alignment, even for aspects such as laws and authors' expectations. For sure, this second edition will not be the last and definitive one for this book: we will see many sentences and regulations written in the near future, and the whole matter is in evolution, but it will still require years to settle.

SM-Tools, Copernicus, FOSS and the reasons of inevitable choices and drifts

2026-02-25T15:00:00Z

Here at work, we develop a series of tools for geospatial processing with multiple goals, expected maintenance durations, different scopes, generalization needs, and motivations. Note that about 10 years ago, here in Europe, a completely new approach to upstream and downstream services for Earth Observation began: ESA changed its data licenses, and distribution and access modalities entered the Big Data era.

That even changed the data approach for academic institutions and triggered a major shift in the daily work of researchers, with access to a tremendous volume of weekly data available almost just in time and worldwide. Of course, such a change also impacted us, and we had to adapt our processing and storage capabilities to the new era.

One of my side projects in that regard is SM-Tools, which consists primarily of a collection of support tools for running our internally developed soil moisture algorithm using SAR satellite data (SMOSAR). One such tool (now named smt_copernicus) began (and evolved with multiple restarts-from-scratch) more than 10 years ago, when the Sentinel constellation started operations. Its purpose is to search for and download satellite products from Copernicus archives using multiple criteria, and to maintain an internal geospatial database of these products, along with all derived maps and ancillary data. This is only one component of a system that should be able to process large quantities of multi-source data on selected areas of interest, create downstream products, and calibrate and analyze results by comparing them with field data. The clear final goal is to achieve new findings in satellite data analysis, supported by extended processing worldwide, and to introduce new algorithms.

This is a long-term goal that unfortunately runs up against short- to mid-term difficulties of accessing archives that are not under our direct control. The sad reality is that in the last 4 years, the Copernicus archive access modality changed 3 times, and in the previous period, Copernicus also changed policies and modalities in progress (e.g., by introducing online and offline products, changing formats, etc.). Geospatial communities are small enough to encounter more practical difficulties than expected in such operational conditions, and this is now an almost weekly experience. We now have to chase other parties’ changes more often than we did in the past, rather than working on our own goals instead.

For instance, until 2023, the main package used for accessing the Copernicus archive was the Sentinelsat Python package (developed by an handfull of willing scholars starting in the summer of 2015). It became abandonware that year when people discovered that the protocol changes required rewriting most of the package from scratch, including all test code and mockings. That happened again at the end of last year. Incredibly enough, all access protocols have been an ongoing affair since 2023, even for non-secondary details, and required frequent adjustments to avoid unexpected breakage in download and processing pipelines. That for sure does not encourage community-supported FOSS solutions. Exactly in 2023, when I discovered our Sentinel-related tool had to be deeply changed because of the lack of the obsoleted Sentinelsat package, I decided that enough was enough and managed to migrate from Python to a fully self-supported Perl reimplementation: one of the things I always hated in the Python ecosystem is the excessive (for me and our purposes at least) speed in deprecation of consolidated packages and features, and the prospective of having to chase after unexpected changes in both Copernicus AND Python sides was out of question. My experiences with Perl have been much less annoying in this regard, with scripts running perfectly even after 20 years since they were written. Let’s consider this the old-school approach: if something is working, don't touch it without a more than valid reason, and even then, think twice before you touch.

In the meantime, around 2019, another independent effort started to support Copernicus access, along with a few other data providers. That’s about 4 years after the original Sentinelsat project. Timing in this case is essential to be considered. EODag is a single-company FOSS product that has been actively developed since then, but may have been considered stable 1-2 years later. Of course, it only provides the usual access layer, and using that package implied, at the time, replacing Sentinelsat with EODag as a base layer for searching and downloading only, while performing other tasks afterward with other self-made code. Before 2023, using Sentinelsat or EOdag was equivalent in order to perform the same task, with very little advantages.

Note that both tools were, in any case, pretty adequate but not enough for our goals, and also had a few defects (or a lack of flexibility, to say better) to manage in some creative way. That has been one of the reasons for not replacing Sentinelsat with EOdag in 2023. The other major one was that the idea of replacing a small package with another (as for Sentinelsat, there are just a couple of main contributors to the codebase, along with a good number of pending issues and PRs for such a kind of product) is probably not the safest one to avoid problems in the near future, when Copernicus will change things (again, see the next Earth Observation Processing Framework EOPF data format — Zarr). And of course, EODag is written in Python, and I already expressed my concerns about that.

Even if you like it or not, nowadays, the concrete alternative to adopting small FOSS projects to perform basic tasks is to use AI tooling to create a perfectly (or almost so) tailored implementation for the target task. While in 2023 I had to rewrite from scratch (in maybe a month of work with some fixes to Geo::GDAL::FFI package for a multi-threading issue) an implementation of a multi-threaded tool for accessing the Copernicus archive and maintaining an internal geospatial database of products consistently among multiple generations of the archive, I implemented the STAC protocol variant in a few hours instead, thanks to Claude Code-based patching and my reviewing and tests for the resulting codebase. As said in this post, currently, the cons of adopting a small third-party FOSS solution outweigh the pros, particularly regarding the resulting technical debt, compared with a well-conducted self-consistent AI-based development process.

I’m seeing in my own side projects exactly the mirror of what will probably be the reality of FOSS projects in the near future, in practice, as I mentioned in the previous post. Relatively few major/interesting projects will be adopted by others and attract contributions, while most codebases will become pure one-man shows, with AI tooling.

A significant part of geospatial processing involves data procurement and processing, i.e., refining and preparing data and images in order to collect, filter, and process large volumes of data for subsequent analysis. This is the most annoying and repetitive part of the process, and also often the most time-consuming. In my experience, working on those tasks is probably the most effective way to use LLMs through a spec-and-test-driven design. Whether you like it or not, it is the most immediate way to produce working code by iterating with a chain of thoughts and accurate reviewing of results, including a decent test coverage. As observed in Antirez's experience, the AI agent also had the nice ability to retain my Perl style (which is not secondary, given that Perl has many programming flavors and variants).

Maybe the final results will be an increase in quick-and-dirty codebases, but for many scholars, it will be a major simplification of their lives. In most domains of science, coding activities have been seen as an inevitable evil: they are a tool, not the primary goal, and even before the advent of LLMs, most scientific codebases were far from something to be proud of. The Copernicus attitude to FAFO will also encourage such approaches. Simply because scholars don't have time to waste chasing changes introduced by this or that data provider or company when contracts change.

AI sloping attitude? No, simple survival instinct.

A decent SSH client for Android is not what one would expect

2026-02-08T16:00:00Z

I’m not too happy with using mobile devices as a daily driver for server connections. When one needs to use a keyboard, in many cases the appropriate device is inevitably a laptop or a desktop computer. Anyway, sometimes it happens that the mobile phone should be used as an SSH client for an emergency or to perform simple remote tasks. Possibly using an app that is usable, decently supported, and can share common configurations among multiple devices with a reasonable level of security.

For years, the most used client on Android has been JuiceSSH, but unfortunately, it became abandonware some years ago, and at the end of last year, it was also delisted from the PlayStore. A few days ago, I changed my smartphone and finally moved to the latest Fairphone model. Now, who knows me already knows how much I hate mobile phones, which are generally the realm of apps with the worst UX ever conceived. Since then, I discovered that the pro option of JuiceSSH is also dead, and basically, SSH forwarding cannot be used anymore. Too bad, I decided to look for a state-of-the-art SSH client application and discovered that the generally suggested apps (i.e., Termius and Connectbot) are the usual PITA.

Thanks gosh, Termux entered the room.

For people who don't know it, it is an Android terminal that emulates a color xterm, but has some nice features, specifically a damn good package management tool built in. Of course, it is pure FOSS.

pkg install openssh git vim

Now it could be nice to access the common storage area by enabling it with

termux-setup-storage

This could be useful for exchanging files with remote hosts and keeping them available for/from other apps.

Now, a useful .ssh/config file can be pulled via git, stored locally, and possibly customized to simplify ssh network access, with all helpful host stanzas. Even a local password-protected host key can be created and stored locally for safety.

So far, so good. Probably, I should simply start giving up on treating Android as a special beast and treat it as just another Linux host. Fewer apps, more terminal, and fuck the majority!

Is AI driven coding the start of the end of mainstream FOSS?

2026-02-04T20:00:00Z

Someone on Mastodon (I’m sorry, but I don’t remember who exactly) published a short post that pointed to a rather technical economic study of the impact of AI on FOSS software development [1]. It is no secret that the AI debate is highly polarized, and the enthusiasts for the current trend in AI applications in the IT domain are at least as numerous as those who are concerned/skeptical. What is certain is that no one can, in the long term, prospectively evaluate the impact of AI on society, particularly in the IT world.

The main thesis of the paper is that AI-based code production will end the mainstreaming of FOSS software, as we have learnt over the last 15-20 years. The paper begins with well-known episodes from recent history (specifically, the Tailwind saga [2] and Stack Overflow's near-death experience [3]).

Of course, the paper presents a theoretical economic model to evaluate a possible impact scenario for the FOSS production model, which could or could not come to fruition, depending on the assumptions made.

My honest opinion is that a conscious and accurate use of AI can accelerate development. That is, in a bad and good sense, I mean directly on the basis of the experience and skills of people who use such models. Therefore, we are both seeing slop and high-profile creations with the aid of AI. Maybe slop contributions are more prevalent simply because mediocre developers are the majority, and mediocrity is the backbone of enterprise production (because it is the most replicable and independent of contributors and their capacities).

Like it or not, modern software industries do not need, and fight against, too much creative approaches. Enterprises need aurea mediocritas, not isolated geniuses. Also, depending on third-party creations, it apparently reduces the enterprise's technical debt because it is typically shifted onto someone else's shoulders. Of course, this is an approach that works until it fails miserably when such a third party disappears, changes its license model, changes its mind about the product, changes its APIs, and so on.

That said, one clear consequence of using AI helpers in coding appears to be the progressive disappearance of many packages, modules, and libraries, which can be easily replaced by AI-generated creations tailored to the task. Just to cite one practical example, Tailwind nowadays could be easily replaced by CSS and simple JavaScript components, with the obvious advantage of not depending on yet another third-party-controlled piece of code that could be subject to abrupt changes from one version to another without notice and break existing codebases. At the same time, Tailwind themes can be generated by AI without even consulting its documentation (which apparently had an immediate impact on the company's revenues).

Another advantage is that AI-based, tailored solutions would reduce the amount of code from external dependencies that solve problems for others, instead of focusing on the minimal set of features for your own needs (with all the implications of possible breakages arising from such an anti-minimalistic pattern).

Of course, using AI helpers in this way does not reduce the effort required to understand and create new software, but it probably raises the required competence to a higher level, which could be better in the long term, while encouraging quick-and-dirty approaches in the near term. The so-called vibe coding is not a black-and-white concept; it has a lot of grey tones directly depending on the awareness, responsibility, and skills of the developer: as said, it can accelerate in many senses - even to crash against a wall - increasing in an uncontrolled manner the technical debt when in the hands of the wrong individual. Even about that, Anthropic recognizes that AI abuse can negatively impact coding skills and debugging capabilities [6].

Add to this the current very high infrastructure load many networks are reporting, for which the AI bots currently seem to be the culprit [4]. This seems like very strange behavior for such botnets, given that web crawlers have been around since the 90s and should be able to handle infrastructure load fairly well by now. It seems that AI companies simply aren't fair enough on their own, or that the training phases of neural nets are definitely more demanding. Maybe both?

So, what do I see as the future for FOSS development as a whole? I am not a pessimist as in the cited article. For sure, I see fewer small contributions in the long term. Today, there is a massive production of AI-slop-based contributions to many prime-time projects, but I see this as incidental. In recent years, GitHub-based path of honors has been a major self-promotion channel for junior developers, which explains the drift toward low-quality contributions: devs are (were?) strongly motivated to contribute and find in AI slop an easy path to that, by creating personal portfolios. That’s also true for fake security-related reports (see the well-known Curl project case [5] and others). This is, of course, annoying, but in my view, that’s the result of current AI hype and should normalize in the mid-term.

Also, in the near future, I see less and less relevance in FOSS projects that are not sustained by a strong architectural idea, innovation-grade, a large community, and a consistent development effort (much bigger than a few weeks or months of work). That kind of project will become mainly background noise, let me say. Maybe that could impact whole categories of FOSS software: it is not a secret that many language hubs are full of packages/modules of dubious quality, often used because they are available just a use/import directive away. In many cases, such products will simply be replaced by an AI-based reimplementation. If the final result will be better or worse in average quality, only the future will show. For sure, the AIAD will cause a progressive democratization/popularization of the development process, giving average users access to possibilities once unavailable to them: we will probably see the production of a plethora of small tools and workflows built on agents rather than finished, refined products, like it or not.

The result could be an increase in FOSS products at the cost of lower average generality and code quality, with a few high-end, tailored products for mainstream applications instead. But was this really so different in the past? I don’t think so. The true difference is probably the increase in quantity in both sets of products, as potentiated by AI tools: if one does not do her/his homework, the result is clearly garbage, but that was true before AI, too.

“AI gives us the worst and the best - simultaneously.”

(Daniel Stenberg, the Curl Mantainer)

References

How to trust FOSS players and the security implications

2026-01-27T17:30:00Z

More and more, recent (and not too recent) episodes [1-5] nowadays show a hard truth we already discovered in the Debian project since the end of the 90s. A key security principle in FOSS code development is ensuring the trustworthiness of all parties involved, and that’s unfortunately also the weakest part of the whole chain.

Debian adopted a long time ago a tentative GnuPG-based screening of people involved in the project through the so-called Web of Trust. All developers need to participate in eyeball meetings to sign each other's public keys after verifying personal IDs. The more signatures, the more trustworthy. That is generally mandatory before being granted the privilege of uploading to the main archive without review. Of course, trust is not automatic, and each volunteer must demonstrate they have the required skills and good intentions, and embrace the Debian social contract. This is the annoying, often multi-year process to be accepted as a contributor.

This process is far from perfect and may also be subject to abuse, as Martin Krafft demonstrated at a past DebConf, a long time ago. One of the main issues is that maintainers work on upstream software that generally does not undergo such a process to accept contributors. As the well-known sad case of the xz utils demonstrated [6] a couple of years ago, an initial review of pull requests is not generally enough to ensure that the developer or group does not have evil intentions in the mid- to long-term. Also, with the best intentions of sane upstream developers, evil parties can make very creative efforts to hide their malware code. This is magistrally demonstrated in that episode, which did not cause major security issues, only by casualties.

The sad reality is that none of the main programming language hubs is really trustworthy, because literally anyone can register anonymously to upload code and participate in development teams. Core teams at least review pull requests before accepting them to avoid major abuses. To address this, enforcing a worldwide web of trust for all core projects and possibly all software hubs should be considered a mandatory step to improve security and accountability.

It does not resolve all problems, but it helps. A central authority is not the answer, as it could create more problems than it solves. Instead, enhance trustworthiness by encouraging ongoing cross-review by multiple parties. Establish processes that build developers' trust over time through active participation and transparent peer review. While key hijacking remains a risk, this is a separate issue requiring distinct protective measures.

I've written about this related issue before. The shift from distributions to language and upstream hubs shifts software management onto developers and users, increasing the risk of security incidents from malicious contributions.

That said, like it or not, most FOSS products out there are created/maintained by single individuals and micro-development teams with no warranties and questionable durability. I wonder how billion-dollar companies can seriously consider basing their core business in such conditions, a problem directly connected to the broader sustainability challenges for FOSS projects. The progressive spread of the AGPL license and other similar licenses is a symptom of this type of malaise and should be taken into consideration, as they are different aspects of the same problem. Security concerns are another key point that we, as a community, should try to manage better, but my honest thought is that nowadays, predatory big (and not so big, even) companies (as well as public bodies too) that use community-driven FOSS code in an unfair manner, without returning a cent for development and maintenance, are not more acceptable.

Therefore, FOSS communities are not perfect, but many of the culprits are nowadays on the shoulders of companies and public bodies that are still looking out their windows instead of being active stewards and promoting reciprocal collaboration among all involved parties.

Come on, put the money and effort into the sources of your digital profits.

References

The perfect desktop is a matter of points of view, or not?

2026-01-22T19:40:00Z

I recently learned about an opinionated flavor of the Arch distribution called Omarchy, which is basically a collection of desktop packages built on top of a rolling Arch distribution. Nothing special, but for the vocal original author of the scripting job at the base of such flavor, who is, as it happens, for many old-school self-centered geeks out there, the quite discussed DHH. I will not enter into the merits of the reasons for the dubious fame of David "DHH" Heinemeier Hansson, which basically stem from some of his past posts on X/Twitter and some of his questionable ideas.

I’m not interested in that here. I’m more interested in some spontaneous thoughts about the hype (well, at least among the very restricted niche group of Linux desktop fans) around this desktop flavor. It is not something new; the Hyprland UX is basically an i3-like tiling window manager with steroids, based on the current non-Xorg incarnation of Wayland, with a few whistles and bells.

I have been a long-time Linux desktop user since the 90s, and a tiling window manager (specifically one of the suckless incarnations, Awesome WM) has been my main desktop for quite a few years. Some years ago, I abandoned such a paradigm when I finally realized that a pure tiling window manager is a great idea until it isn't. Basically, most of its pros (one application per virtual desktop, easy tiling on big displays, keyboard-driven navigation) can be easily replicated in a capable desktop environment like any current Gnome version. This has the big advantage of being ready for use right after installation and of being easily and fully customizable via plugins. The cons of a tiling WM are always present, based on workflows, and there are generally no easy workarounds. The biggest is the need to find tricks and third-party tools to solve use cases that are not always trivial (or worse, that are trivial on a DE instead).

A DE has the indisputable advantage of including all batteries for widgets and customization tools, whereas most (all) WMs require third-party tooling to manage many disparate configuration snippets, such as Bluetooth, Wi-Fi, hot-plug devices, auto-sensing of beamers, dynamic multi-display, fast binding of container apps, accessibility featues, and many others. Too often, also, such WMs require using a command-line tool or a workaround to perform tasks that are simply part of the common DE experience.

I also remember the pain of using the multiwindow GUI of GrassGis under Awesome, which was at the time just another type of application under a floating window manager, instead. When an application opens a new window for every new module used, well, the UX could become a nightmare under a tiling WM, if you are not using a 43-inch display. The same goes for virtualized desktops, too: when the guest and host collide for keyboard use, continuous control switching can rapidly lead to madness. That’s just a pair of examples to conclude that the coolness of a desktop implementation is often a matter of perspective and personal workflows, and I constantly found that a mandatory tiling WM paradigm is simply less flexible in some practical cases.

To be honest, I find the Omarchy UX to be the typical incarnation of a canonical WM-based interface for fresh Linux desktop users. Such users are divided into two classes:

The class of people who are searching for an exact replica of Windows/macOS GUI. A no-hope group of people: if something has to look like Windows, have exactly the same policies and the same applications and icons, even, well, probably they should stay with Windows: simple and clean. They are the most critical and vocal complaining users for whom the Linux-on-the-desktop era will never come.
The class of people who look for something radically different and discover the keyboard-driven interfaces as something almost magical (without fully realizing that such an experience can be replicated easily and mostly by using environment shortcuts and some simple plugins). A trivial secret, I would say. They are the most enthusiastic about this kind of desktop, but also regular distro-hoppers (yes, it is an offense for me: distro-hopping is for gamers, not for workers who need to complete primary tasks daily) from time to time; more often, they will never admit they are simply playing around, and solving auto-inflicted problems is part of their game.

Of course, the WM-based desktop paradigm still has its own use cases, which I group under a very few limited cases:

You are using a strictly roll-on distribution, such as Debian sid/unstable, Arch, Guix, Gentoo, and many other in-development flavors of other mainstream distributions, including Fedora and OpenSUSE. On such distributions, avoiding desktop environments reduces the likelihood of encountering temporary problems after daily upgrades, as some transient (in the order of days/weeks) breakages can occur. But who is the user of such platforms today? Seriously, I think only someone actively involved in development and testing should be interested in such distributions. Today, most desktop apps are distributed as containerized packages via one of the multiple available hubs for Flatpak, AppImage, Snap, Docker/Podman. I can’t see the practical advantage of using an unstable distribution on a daily basis. If you think differently, dude, you have a problem, and it is not the distribution you are using, but it is what you see in the mirror. If you are not a YouTuber who needs to produce videos to monetize, well, you are probably simply using the wrong distribution pointlessly (and creating your own problems from time to time, which are perfectly part of the roll-on experience, by manual).
You are using an old, low-resource box with limited RAM and cores. A platform that cannot simply be used with the current desktop environments. I seriously doubt it could also be used for general computing, indeed. Nowadays, even a web browser is simply a hungry hog on such platforms. I mean, a dual-core box with 4 GB of RAM that could be more than 15 years old. If this is your platform, well, a window manager is perfectly legitimate, but it probably couldn't be used with Hyprland either. But to do what? I mean, but for installing it and telling it to friends...
You are an old-style lazy geek, anchored to your own configurations, refined over dozens of years, with very few reasons to change. That’s perfectly legitimate, but most of those configurations could probably be out of date. I know, you are still adjusting your Modelines in your Xorg configuration. Well, dude, probably it’s time to come down the tree in the forest.

Of course, I also tried to install Omarchy on an old box of mine (an 11-year-old Lenovo ThinkPad L540 with a dual-core i5 and 8GB RAM) that runs perfectly with the current Debian 13 and Gnome 48. Sadly, it was not even booted to install: just a dark screen. Good, but not too good, dudes.

And this leads me to the elephant-in-the-room argument for this post. Most users need stability and, occasionally, up-to-date applications. The average users need certainty that they can easily install an OS on most platforms and have a stable UX for a decently long period (let’s say 2-5 years without any reinstallation in between). The more users, the more stability. The simpler, the more effective, too. And that’s the real point most devs (or wannabe experts) have probably missed in the meantime. The desktop is a mere tool; it should not require an expertise addiction.

It is not a matter of DE vs WM, but of homogeneity and generality versus good, but not enough for all. If one has to rediscover the warm water to manage a configurable tool that, in a DE, is a click-n-point away, it is a failure in general UX. Of course, even DEs are far from perfect, but too often, WM UX is far from even being basically complete.

For instance, I can easily manage my full clipboard history with inter-session persistence thanks to a simple Gnome plugin (i.e., Clipboard Indicator). There is no equivalent widget in most WMs, but they need to use a third-party tool to provide something that is almost equivalent, but often incomplete. Well, Houston, we have a problem! That’s just an example, but the general approach is clear: if one has to constantly sacrifice immediate, good-enough implementations to adopt half-finished tools or workarounds to solve basic GUI workflows, WMs become not accelerators of productivity but defective implementations, and that has been my constant experience in that regard with WMs. At some point, one has to set priorities, and after years, my priority has become not to waste time reinventing the wheel for desktop GUIs. Sorry, guys. There is more than one way to implement a desktop interface, but many of them can simply become a pain because they are not flexible enough or incomplete, resulting in continuous adjustings and workarounds to have something decently working.

And yes, this is another damn opinionated post about the current Year of Linux on Desktop. Don't take it too seriously...

A Terramaster NAS with Debian, take two.

2026-01-19T13:00:00Z

After experimenting at home, the very first professional-grade NAS from Terramaster arrived at work, too, with 12 HDD bays and possibly a pair of M2s. NVME cards. In this case, I again installed a plain Debian distribution, but HDD monitoring required some configuration adjustments to run smartd properly.

A decent approach to data safety is to run regularly scheduled short and long SMART tests on all disks to detect potential damage. Running such tests on all disks at once isn't ideal, so I set up a script to create a staggered configuration and test multiple groups of disks at different times. Note that it is mandatory to read the devices at each reboot because their names and order can change.

Of course, the same principle (short/long test at regular intervals along the week) should be applied for a simpler configuration, as in the case of my home NAS with a pair of RAID1 devices.

What follows is a simple script to create a staggered smartd.conf at boot time:

#!/bin/bash
#
# Save this as /usr/local/bin/create-smartd-conf.sh
#
# Dynamically generate smartd.conf with staggered SMART test scheduling
# at boot time based on discovered ATA devices

# HERE IS A LIST OF DIRECTIVES FOR THIS CONFIGURATION FILE.
# PLEASE SEE THE smartd.conf MAN PAGE FOR DETAILS
#
#   -d TYPE Set the device type: ata, scsi[+TYPE], nvme[,NSID],
#           sat[,auto][,N][+TYPE], usbcypress[,X], usbjmicron[,p][,x][,N],
#           usbprolific, usbsunplus, sntasmedia, sntjmicron[,NSID], sntrealtek,
#           ... (platform specific)
#   -T TYPE Set the tolerance to one of: normal, permissive
#   -o VAL  Enable/disable automatic offline tests (on/off)
#   -S VAL  Enable/disable attribute autosave (on/off)
#   -n MODE No check if: never, sleep[,N][,q], standby[,N][,q], idle[,N][,q]
#   -H      Monitor SMART Health Status, report if failed
#   -s REG  Do Self-Test at time(s) given by regular expression REG
#   -l TYPE Monitor SMART log or self-test status:
#           error, selftest, xerror, offlinests[,ns], selfteststs[,ns]
#   -l scterc,R,W  Set SCT Error Recovery Control
#   -e      Change device setting: aam,[N|off], apm,[N|off], dsn,[on|off],
#           lookahead,[on|off], security-freeze, standby,[N|off], wcache,[on|off]
#   -f      Monitor 'Usage' Attributes, report failures
#   -m ADD  Send email warning to address ADD
#   -M TYPE Modify email warning behavior (see man page)
#   -p      Report changes in 'Prefailure' Attributes
#   -u      Report changes in 'Usage' Attributes
#   -t      Equivalent to -p and -u Directives
#   -r ID   Also report Raw values of Attribute ID with -p, -u or -t
#   -R ID   Track changes in Attribute ID Raw value with -p, -u or -t
#   -i ID   Ignore Attribute ID for -f Directive
#   -I ID   Ignore Attribute ID for -p, -u or -t Directive
#   -C ID[+] Monitor [increases of] Current Pending Sectors in Attribute ID
#   -U ID[+] Monitor [increases of] Offline Uncorrectable Sectors in Attribute ID
#   -W D,I,C Monitor Temperature D)ifference, I)nformal limit, C)ritical limit
#   -v N,ST Modifies labeling of Attribute N (see man page)
#   -P TYPE Drive-specific presets: use, ignore, show, showall
#   -a      Default: -H -f -t -l error -l selftest -l selfteststs -C 197 -U 198
#   -F TYPE Use firmware bug workaround:
#           none, nologdir, samsung, samsung2, samsung3, xerrorlba
#   -c i=N  Set interval between disk checks to N seconds
#    #      Comment: text after a hash sign is ignored
#    \      Line continuation character
# Attribute ID is a decimal integer 1 <= ID <= 255
# except for -C and -U, where ID = 0 turns them off.

set -euo pipefail

# Test schedule configuration
BASE_SCHEDULE="L/../../6"  # Long test on Saturdays
TEST_HOURS=(01 03 05 07)   # 4 time slots: 1am, 3am, 5am, 7am

DEVICES_PER_GROUP=3

main() {
    # Get array of device names (e.g., sda, sdb, sdc)
    mapfile -t devices < <(ls -l /dev/disk/by-id/ | grep ata | awk '{print $11}' | grep sd | cut -d/ -f3 | sort -u)

    if [[ ${#devices[@]} -eq 0 ]]; then
        exit 1
    fi

    # Start building config file
    cat << EOF
# smartd.conf - Auto-generated at boot
# Generated: $(date '+%Y-%m-%d %H:%M:%S')
#
# Staggered SMART test scheduling to avoid concurrent disk load
# Long tests run on Saturdays at different times per group
#
EOF

    # Process devices into groups
    local group=0
    local count_in_group=0

    for i in "${!devices[@]}"; do
        local dev="${devices[$i]}"
        local hour="${TEST_HOURS[$group]}"

        # Add group header at start of each group
        if [[ $count_in_group -eq 0 ]]; then
            echo ""
            echo "# Group $((group + 1)) - Tests at ${hour}:00 on Saturdays"
        fi

        # Add device entry
        #echo "/dev/${dev} -a -o on -S on -s (${BASE_SCHEDULE}/${hour}) -m root"
        echo "/dev/${dev} -a -o on -S on -s (L/../../6/${hour}) -s (S/../.././$(((hour + 12) % 24))) -m root"

        # Move to next group when current group is full
        count_in_group=$((count_in_group + 1))
        if [[ $count_in_group -ge $DEVICES_PER_GROUP ]]; then
            count_in_group=0
            group=$(((group + 1) % ${#TEST_HOURS[@]}))
        fi
    done
}

main "$@"

To run such a script at boot, add a unit file to the systemd configuration.

sudo systemctl  edit --full /etc/systemd/system/regenerate-smartd-conf.service
sudo systemctl enable regenerate-smartd-conf.service

Where the unit service is the following:

[Unit]
Description=Generate smartd.conf with staggered SMART test scheduling
# Wait for all local filesystems and udev device detection
After=local-fs.target systemd-udev-settle.service
Before=smartd.service
Wants=systemd-udev-settle.service
DefaultDependencies=no

[Service]
Type=oneshot
# Only generate the config file, don't touch smartd here
ExecStart=/bin/bash -c '/usr/local/bin/create-smartd-config.sh > /etc/smartd.conf'
StandardOutput=journal
StandardError=journal
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

AI training, copyright and the future of contents creation

2026-01-11T21:00:00Z

I have already addressed the implications of modern LLMs, specifically their training, in the context of copyright and licenses for both code and original content. A 'IANAL' disclaimer applies to this post, but my honest opinion is that such training is a legitimate type of reading and learning after study, unless explicitly excluded in licenses among the licensee's rights.

Following the exploitation of LLMs and the AI boom that began in 2022, several lawsuits and litigations emerged among multiple parties, with a few reaching a significant milestone through the first court rulings. Note that every country has a bit different regulations about copyright and fair use, so the current lawsuites could be only the starting point of a long list of legal actions.

While most of the current lawsuits seem to demonstrate that Anthropic or Meta had the right to use books bought (in paper or digital form) for LLMs training (on the basis of the fair use principle), the most problematic aspect instead is the apparent use of pirated books taken from LibGen and other known piracy websites, which - if confirmed - can result in potentially destructive damange for the companies, to compensante authors and pay fees in the order of hundreds of billions.

The same problems are present in the coding parts: again, using FOSS-licensed code for training could fall under fair use, but training using private codebases, as well as proprietary ones, could be equally destructive for the same companies, as well as for GitHub and Microsoft. The key point would be demonstrating, without any doubt, the unfair use of private or pirated content, of course.

Of course, I'm quite sure future licenses for FOSS codebases and documentation could include an explicit exclusion clause for AI training, which could jeopardize the legitimation of use even for future FOSS code. I would expect such a license change, as some projects already explicitly exclude AI-based contributions. My opinion about such a question is that it could represent shooting oneself in the foot, due to the pervasivity of AI tools among developers currently. Adoption of AIAD could represent a boost in development time if adopted with a healthy dose of skepticism (i.e., a human-in-the-loop approach). About that, I'm quite convinced of Linus Torvald's point of view: the point is not who writes the code, but who is technically responsible for it and ensures the required quality review and supervision.

Moreover, an implication of the current polarization in the AI hype is the future (present?) crisis of traditional web content providers. A symptomatic case is the StackOverflow crisis, which will, with high probability, lead to the end of the service as we know it in the near future.

That will have an impact on future AI training, too, for sure, because SO has been for years a huge source of knowledge about multiple fields in IT. What if fewer and fewer people will contribute to Wikipedia and general web content? What if more and more sources of information were to reserve the right to use their information for pure human-driven study? Knowledge has not been static in human history; AI models will need to continuously enrich their training sets and stay up to date.

It would be grotesque if the whole AI hype were brought to a halt by such copyright-based legal questions (even if I'm pretty sure a fully fair training would be possible now for such companies, who knows the impact of a more limited approach on the final result?). Surely, this seems the most serious threat to the future of such companies and the whole AI-based solutions.

The only true solution to such a threat is finally having a true open training model, which details sources and the whole process of training with full transparency, something that even the so-called open AI models are still far to be ready to provide.

References

This was for every one: about the crisis of the web

2025-12-25T15:30:00Z

I just finished reading the delightful book by Sir Tim Berners-Lee, titled This is for Everyone, published this year. It is a trip, long, almost 400 pages, about the origin and evolution of the World Wide Web, seen by those who conceived and pushed it from the start. The entire first part of the book is dedicated to the history of the web, the W3C, and the Web Foundation's operations as we have known them in the first 30 years of its development, from 1989 onwards.

I was there at the very beginning of the 90s: I was connected to the Internet since 1991, and reading such a book for a good part has been an emotional trip in my memory of those events and people. He is a visionary and an idealist who fought for an extended period to prevent his WWW creature from being intercepted and disrupted by for-profit interests.

It happened almost from the start, when first NCSA, then Netscape, and Microsoft tried one after the other to change the whole idea of openness into something proprietary, driven through the same scheme of embracing, extending, and extinguishing. In practice, the complete negation of standard and openness, with a clear goal in mind: obtaining users' lock-in into proprietary products, clearly for profit.

Tim provides evidence on multiple critical aspects of the current incarnation of the net as we know it today and over the last 20 years or more. They are both technical and social defects or drifts. The web is no longer what we learnt to know in its first years of existence. The start of the end of the original web concept was the mobile-first approach, which relegated the use of a regular computer to a second-class experience for most users. Most of the digital-native people never used a computer to access the network, and that user experience deeply affects the current vision of the web.

For years, nowadays, a browser has not been the main program for accessing content and services. Social networks are mostly not interoperable because companies have little interest in having their users leave the walled gardens of their apps. Using a browser and potentially exiting the company's services to access other servers and spaces is tolerated, but is perceived as damaging profits. That's simply because users are not users, but customers. The result is the shattered Internet about which I already wrote: the W3C standards are still relevant, but embedded in applications and frameworks that enrich and upset the user experience with proprietary workflows and extensions.

An emblematic case is Apple, which has, in practice, abandoned its WebKit engine and Safari browser in favor of apps and proprietary services to monetize customers and companies.

The concrete risk is that the whole web and its standards would become a marginalized component of the net, while most users are confined to walled-off realms of proprietary services and social networks. The recent AI innovation can mark the definitive end chapter of web content creation and search as we have used them over the last 30 years. More and more users will limit themselves to AI-provided overviews instead of collecting and consulting multiple sources of information and independent services. That will also have a concrete impact on revenues and interest in content creation and provision at large.

The second part of the book is fully dedicated to all such problems: the impact of social networks, the last few years of generative AI, the BigCo dominance, and includes all Tim's worries for the foreseeable future. He's an idealistic, optimistic, and positive guy due to his past experiences. However, he also has a good dose of sane realism. He understands that the path is nebulous and full of dangers (specifically, the AI path is highly polarizing and can hide multiple issues at many levels).

He sees in the indie web, and specifically in open and well-structured distributed standards (such as the ActivityPub protocol), a possible way to change the present and future by favoring interoperability and independence. A concrete proposal is the Solid standard for personal data wallets (or pods in Solid terminology) under complete user control for accessibility by third-party services. Such a standard is still in its infancy, but the true problem I see is the trustworthiness of involved parties, both companies and governments. Trusting is the key, and maybe we all individually lost such a superpower a long time ago.

Creating a corpus of rules to manage all such technologies and ensure ethical behavior can be a desperate illusion; the only concrete alternative would be to opt out, at the cost of exclusion from the social context (not only the digital one). But I agree there is no other way to recover the original idea of the web. The AI technologies are even more polarizing, among doomers and boomers, with a bumpy road ahead. For sure, open protocols and distributed multi-peer services are the inevitable starting point, but they won't be enough.

"It was not enough simply to release new technology and hope for the world to improve. You had to develop technology and society together. You really had to fight, in a principled and continuous way, for human rights. The web offered people a platform for their voices to be heard, reducing the cost of publishing and distributing information to effectively nothing. But, used improperly, it could also be turned into a tool of surveillance and control." (timbl)

References

Creating your own Guix substitute server

2025-12-21T16:30:00Z

I lately dedicated some time to setting up my own substitute server for Guix on a foreign distribution. This post is about that experience, after verifying that such a process is currently quite underdocumented. A substitute server is clearly a required step in order to cultivate a personal or unofficial/alternative channel for Guix, at least if one has more than one box (and possibly one physical location) to manage.

For people not up to date with the news from these digital bits, in the last year or so, I elected the Guix package manager and system as my second-preferred OS, after Debian GNU/Linux. I published more posts about it, all readable under my dedicated feed.

What is a Substitute Server?

Guix is mainly a source-level roll-on distribution, so at every pull, there is a concrete risk of having to build your own binaries if no binary provider is available to speed up your update process. That is a certainty if you play with your own packages. This is the concrete reason to possibly have a substitute server in the LAN, even though the official ones can be subject to problems and DDoS. Specifically, I set up a substitute server based on Debian GNU/Linux reachable both at home and at work. While the Guix manual is currently quite verbose about how to use a substitute server from a client, it is bit incomplete about the server side.

Installation of Guix on Debian

The very first step is to install the Guix package manager on Debian, which is currently not more possible using a Debian package from the main archive, as in the past (see this LWN post about). Currently, a binary provided by the Guix project has to be used for that. It is quite straightforward if you use the installer script provided with some caution.

For instance, to install the current Guix version, it is possible to run:

$ wget -O /tmp/guix-binary.tar.xz \
    https://ci.guix.gnu.org/search/latest/archive?query=spec:tarball+status:success+system:x86_64-linux+guix-binary.tar.xz
$ wget -O /tmp/guix-install.sh https://guix.gnu.org/install.sh

# do your homework to check and verify binary and script, then...
$ sudo -i

# export GUIX_BINARY_FILE_NAME=/tmp/guix-binary.tar.xz
# chmod a+x /tmp/guix-install.sh
# apt install uidmap gnupg
# /tmp/guix-install.sh
# rm -f /tmp/guix-install.sh /tmp/guix-binary.tar.xz

That will install a daemon for building sources under an unprivileged guix-daemon user, which has full control over the guix data store at /gnu/store.

Daemon configuration and storage

Such a daemon is run under systemd control with a specific unit file. First, the Guix data store needs plenty of disk space and inodes; I suggest moving the /gnu/store tree under an XFS or even a BTRFS filesystem, for that.

Second, the current installer tries to mount /gnu/store in read-only mode, which can prevent running the guix command as an unprivileged user on foreign distros. This can be disabled if problems occur, as could happen under specific conditions. Once the guix-daemon service is stopped, the gnu-store.mount can be disabled by ensuring the /gnu/store directory is correctly assigned to the guix-daemon user:

$ ls -ld /gnu/store/
drwxrwxr-t 584 guix-daemon guix-daemon 12M 18 dic 07.31 /gnu/store/

The systemd unit file for the Guix daemon

This is the proposed unit file for the guix-daemon process:

[Unit]
Description=Build daemon for GNU Guix
# Start before 'gnu-store.mount' to get a writable view of the store.
Before=gnu-store.mount

[Service]
ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon \
  --discover=yes \
  --substitute-urls='https://bordeaux.guix.gnu.org https://ci.guix.gnu.org'
Environment='GUIX_STATE_DIRECTORY=/var/guix' 'GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale' LC_ALL=en_US.utf8
# Run under a dedicated unprivileged user account.
User=guix-daemon
# Bind-mount the store read-write in a private namespace.
PrivateMounts=true
BindPaths=/gnu/store
MountFlags=private
Type=exec
AmbientCapabilities=CAP_CHOWN
StandardOutput=journal
StandardError=journal
OOMPolicy=continue
Restart=always
TasksMax=8192

[Install]
WantedBy=multi-user.target

Note that, when building from sources in specific cases, it could be a good idea to provide an additional TMPDIR environment variable to bypass the limited size of the /tmp directory (specifically for tmpfs implementation as for Debian 13+). For instance, building a full Linux kernel could require more than 32 GB of storage available in /tmp, which corresponds by default to at least 64 GB of RAM on the system. A typical personal box can be quite limited for that. So adding TMPDIR=/var/tmp to the Environment variable could be a sane idea.

Setting up Guix Publish

To set up the substitute server, a guix publish process must be started via systemd. First, create a publishing asymmetric key pair and a destination for the cache directory:

$ sudo -i guix archive --generate-key
$ sudo mkdir -p /var/cache/guix/publish
$ sudo chown guix-daemon:guix-daemon /var/cache/guix/publish

This is the systemd unit file for the guix-publish.service:

[Unit]
Description=Guix substitute server
After=network.target

[Service]
Type=simple
User=root
Environment='GUIX_LOCPATH=/root/.guix-profile/lib/locale'
ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix publish \
  --listen=0.0.0.0 \
  --port=8080 \
  --user=guix-daemon \
  --compression=gzip:9 \
  --compression=lzip:9 \
  --cache=/var/cache/guix/publish \
  --ttl=30d --negative-ttl=1h \
  --workers=4
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

This will start the service in unprivileged mode on an unprivileged port, which can be easily routed via an NGINX proxy, for instance (my preferred way). I use this system to route the service on both the home and the at-work LANs.

Once done, of course, the next point is defining a policy to populate such a server. While it could be nice running a proper cuirass build service to run continuously building from a custom repository, it is also perfectly fine simply using a manifest file with common packages or the multiple configurations used by personal boxes. Here is a series of alternatives to build a handful of packages.

guix build gcc
guix system build my-laptop-config.scm
guix build -m manifest.scm
guix build make cmake vim emacs perl

A simple manifest example:

(specifications->manifest
 '("gcc-toolchain"
   "python"
   "emacs"
   "git"
   "make"
   "cmake"
   "perl"
   "grass"
   "gdal"
   "geos"
   "gawk"
   "vim"
   ;; Add more packages...
))

Of course, a very simple strategy could consist of running regular pulls & builds for all packages of primary interest and making frequent enough purges of the data store to avoid excessive storage consumption. That can be avoided if plenty of space is available for builds.

Once setup the server and populating it, it is possibile to add its URL to the configurations of all personal/LAN boxes for foreign and native installations, and live peacefully.

Happy guixing, folks!

About computing environments for reproducible science

2025-12-09T13:00:00Z

A few weeks ago I gave a lecture for the Spatial Ecology course to introduce a handful of junior and not-so-junior researchers from various domains to the not-so-nice world of scientific computing environments.

For people interested, here are my slides about this topic. They are somehow specialized for the Python ecosystem (which has become nowadays the programming language adopted for scientific computing in multiple contexts), where, in the last few years, a lot of evolutions have taken place for the management of dependencies and the management of the computing environment. This problem is amplified in the HPC context (I already wrote a semi-serious post about such an argument).

I also cited guix without more details (it was impossible to deal with all sub-topics in the lecture, and I know that multiple listeners already had problems fully understanding the matter).

Reasoning about that, it is not a silly idea to write some blog notes about the whole topic. First of all, what is the context? Reproducible science is not a novel matter. Any scientific experiment should be reproducible, starting from the same data and giving comparable results: this is the basis of the scientific method (Galileo docet). In the context of scientific computing, that implies that the whole execution environment should be fully reproducible in order to ensure the possibility of replication of the executions, with the same outputs starting from the same inputs. Possibly later on, running on the same platform, or after deployment on a new, completely different system.

The key point is that the long-term reproducibility of such results on current platforms and with current languages is minimal, to be generous. Having the full source code of a Python notebook, a git source repository, or anything comparable is simply only the starting point. The sad reality is that in practice, the source code has, in too many cases, a lifetime of a few months because of the understimation of such a problem by the average scholar. When following a few good practices, such a lifetime can be extended to a few years, maybe.

When I wrote my thesis, too many years ago, I developed the whole C source for execution on a parallel computer of the time. It was a Meiko Computing Surface, a SIMD platform based on INMOS Transputers. The C code used a proprietary message-passing library, CSTools, to enable communication among T-800 processors (unfortunately, there is no relation with the Terminator series, sorry). Now, it is somewhat expected that a code based on a dead proprietary library running on a dead hardware platform could have reproducibility issues today, after more than 30 years.

What is unexpected is that one could have the same reproducibility problems after 30 months, or in some limited cases, after 30 days. I mean both at the binary and source levels, often. Now, part of the problem is due to the FAFO attitude of some development communities. Not all teams are like the GDAL one, which is capable of maintaining the same well-refined APIs for dozens of years. More usually, from time to time, new versions of libraries and tools introduce expected or unexpected breakages against past versions and APIs, which backfire on programs that use them. In other cases, new versions can fix and/or introduce bugs of primary interest for dependent software. Those are the main reasons to meticulously annotate and document every single version of direct and indirect dependencies. This is somewhat solved by dependency resolvers, as explained in my lecture. But that's only part of the whole chain.

Unfortunately, nowadays this chain of dependencies traverses a single language and crashes against system-level dependencies, including the whole operating system, with its system compilers, interpreters, and libraries. This problem is amplified in a fully containerized world, which is nowadays used intensively. Depending on a third-party-provided binary image taken from any hub out there is not a safer approach. Such images can disappear from night to day or have a limited lifetime, so the conscious scholar should also develop his/her own from scratch, which often is particularly out of the skill perimeter of the average scholar.

This is exactly where Guix tries to give an answer. Guix is a source-level package manager with a set of full descriptions written in Guile Scheme for the whole chain of dependencies up to the kernel level. Combining such an analytical description of the system for any built artifact in the timeline from the starting point (derivations), along with the possibility to use build systems to cache binary artifacts (substitutes), and install any software at the user level, does allow the creation of a source-level definition of a full execution environment.

Such an ambitious goal is not without problems, as magistrally summarized by Ludovic Courtès here, but anyway it is at light-years of distance from the possibility of the average deployment system that needs instead continuous babysitting in order to ensure a working environment.

What is probably of interest for general consumption would also be a consistent additional security tagging for derivations in order to fast identify sources with known CVE-tagged versions in the chain of dependencies. That would increase the level of self-awareness when the Guix time machine is used to go back in the past and pick some sources from Pandora's box. It would also be of considerable interest in the SBOM context outside the perimeter of science computing.

So, guix is not perfect, but again a sure advancement towards reproducible computing environments, which currently lack in a way or another in the science domain (and not only that).

References

[1] A guide to reproducible research papers

[2] Guix as a tool for reproducible science

[3] Using Guix for managing reproducible, flexible, and collaborative environments in a PhD thesis

[4] Reproducible genomics analysis pipelines with GNU Guix

[5] Replication crisis

Too many eyes or too few efforts?

2025-12-07T16:00:00Z

I recently read a post by Jack Poller about the end of FOSS optimism in creating software in recent years. His thesis is that the myth that the more eyes that look at a piece of software, the higher its quality, is indeed a myth, and that nowadays it is also a dangerous illusion when we concentrate the analysis on security. Commercial software, on the other hand, has processes and resources dedicated to managing security, which in these times of active AI use could make the difference.

I agree with such a thesis at large, but with some differences in declination and sense. It is absolutely true that security requires special skills that most developers do not have and (above all) are absolutely not interested in having. This is amplified in the FOSS communities, where many programs and libraries are not specifically written with security in mind. On the other hand, currently most AI-generated security reports are fake and pure hallucinations (e.g., ask the Curl maintainer, about the overwhelming number of reports in the last few years that are clearly generated with help of AI and are pure garbage). Therefore, having new AI-based tools for screening does not simplify the problem neither for software creators nor for infosec experts. Basically, they still need to do their time-consuming jobs with resources and efforts: there is not an easy escape for that.

Even, if you like it or not, nowadays FOSS software production is organic to the commercially supported one, and they are strictly connected. If the security (and, more generally, the sustainability) of so many FOSS projects today is a problem, it is also a problem for their commercial counterparts. In other words, FOSS development has been mainstream for at least the last 20 years: that implies that the sustainability (and security) problem extends to the entire supply chain for both FOSS and commercial software.

I already dealt with some of those topics in past posts (e.g., have a look here, or here), not referring to security, but to the sustainability of the FOSS ecosystem of independent-but-interdependent projects. This is a matter of responsability for both FOSS mantainers and companies, and not something that could be avoided as a problem for others.

As already written in the past, I can't see a recipe for success, but I'm quite sure that overcomplication of information systems does not help, but is part of the problem. Too many dependencies, complex frameworks, and architectures are sure highways for disasters. And that's true for both FOSS and commercial software. The shattering of code hubs among multiple sources of binaries, packages, images, and distributions is another sure quarry of problems, because they imply a multiplication of possible origins of issues, as well as the need of sustaining multiple teams and replicating efforts instead of concentrating them.

There is great chaos under heaven; the situation is excellent! (Mao Zedong) .

DebianGis anniversary and the power of being a community

2025-10-16T18:15:00Z

A few days before today, 21 years ago, I sent this message to the debian-devel-announce mailing list to solicit helpers in packaging and to oversee the geospatial software stack included in the main Debian archive. After so many years, still there.

At that time, the Debian project had already been around for more than 10 years and even had a few releases behind, but the typical maintenance of software was still a one-person show, except for a few large and complex pieces of software. Even the kernel was managed by a single developer, Herbert Xu. Each developer was responsible for implementing updates, fixing bugs, and releasing updates. Often jealous of such prerogatives and their stated possession of the package or task. While there was already a quality assurance team and an orphaning process for abandoned packages (or inactive developers), these processes were not very common and were even relatively slow and imperfect.

Since that time, team-based maintenance has become the standard approach in the Debian community for properly managing the most complicated software collections. That's at least to ensure proper long-term maintenance, because the average developer can provide only a limited continuity to their efforts, and real life is generally more complicated than the digital one. It is not secondary that packaging tasks can be tedious in the long term, and it is easy to lose motivation. The presence of a working team can reduce the risk of burnout and allow each developer to step down when needed. The most effective team should be small enough to coordinate more easily and avoid lagging in changes and migrations, but not too small: presenting a one-person show as a team work is not a great idea. But too many people in the same team is equally not a great idea for the opposite reason.

In the specific case of Debian, the system is so modular and lacks many interdependencies that it favors the creation of fully independent management groups for hundreds of components and ecosystems of packages. It is not casual that most of the bugs and inconsistencies are concentrated where too many parts need to interact appropriately with perfectly aligned programming interfaces.

Talking about DebianGis and the geospatial software, the key motivation at the time was the lack of a coordinated effort to build and collect together, with consistency, a lot of different libraries and programs that were (and still are) specialized enough and based on hundreds of dependencies, often not within the perimeter of competence of the geodata user. At that time, piling up the software stack of a typical geospatial application was not an easy task for the faint of heart, and most (all?) of the Linux distributions lacked in one way or another.

Anecdotally, I can remember about 15 years ago, the company that sold us a Suse Enterprise-based solution for a geospatial information system that had so many problems in completing the required setup that I finally created a chroot-based environment with a Debian stable plain install to run a working PostGIS DBMS. That was the time when containerized solutions were still far from being supported, so the chroot environment was the most immediate solution to such a problem. A little win for a community-based distribution and its tiny geospatial team, which provides a measure of the problems at the time. A giant step for the whole FOSS concept.

I'm currently much less active in packaging tasks, but still seeing the current team alive and capable of releasing well-supported products more than twenty years later gives me reason to be proud of such a community.

Installing Debian on a USB stick for a Terramaster NAS

2025-10-15T18:00:00Z

I recently bought a basic NAS for home use. The NAS is a nice Terramaster F2-425, which is a very basic RAID1-only NAS with a decent CPU and 2.5Gb network. Terramaster allows users to either use its custom Linux-based TOS or install any other operating system supported by the x86_64-based platform. Note that this model does not mount any NVME unit for the OS, as for the F2-424.

Common choices include TrueNas, Proxmox, or any other Linux-based distribution. My choice has been a plain Debian stable distribution because I do not have special requirements and prefer a lightweight CLI-only solution over a dashboard. The F2-425 does not have NVME cards, only regular HDDs/SSDs. However, when installing an independent OS, as in my case, you can immediately use an external USB stick for the system, and dedicate HDDs to data. The unit even has a tiny (264MB) internal USB stick for installing TOS, but I simply used a decent 16GB SanDisk thumb drive. The clear advantage is the possibility of having the base system and data perfectly separated, and multiple copies of the stick for safety.

Of course, the installation of such a system can be done without using the Debian installer at all, so I'm describing here how to perform such an installation for my future reference and for other geeks. Of course, you need a running Linux system with debootstrap installed. The process involves partitioning the stick in GPT mode, installing the base system and EFI, and configuring the system to finalize a bootable system with the necessary software to connect the NAS to the network, including OpenSSH.

Note that the 2.5 Gb Ethernet is a RealTek, so a firmware blob (firmware-realtek package on Trixie) is required to properly work with that. Alternatively, another of the USB ports could also be used to add a wireless connection. The OS stick could be simply mounted on the internal port, but it requires opening the chassis for that and using a tiny stick.

At power-on, the internal TOS dongle automagically boots up, so connecting an HDMI display and a keyboard is required to change the setup to boot the Debian EFI image on the stick. On F2-425, press the <F12> key to access the AMI setup and change boot priorities. There are always slight differences among AMI BIOS setups, so it is required to find the right key to access settings and change boot options.

Let's consider /dev/sde as the name of the USB stick device on the host where it will be prepared. A GPT partition can be created via GNU parted, as follows:

parted /dev/sde 	# to create EFI and root primary partition
partprobe --summary /dev/sde
sfdisk -l /dev/sde
mkfs.vfat -F 32 /dev/sde1
mkswap /dev/sde2
mkfs.ext4 /dev/sde3

Once done, installing the base system is immediate.

mount /dev/sde3 /mnt
mount /dev/sde1 /mnt/boot/efi
debootstrap trixie /mnt

mount -o bind /dev /mnt/dev
mount -t devpts devpts /mnt/dev/pts
mount -t proc proc /mnt/proc
mount -t sysfs sysfs /sys /mnt/sys
mount -t tmpfs run /mnt/run

cp /etc/apt/sources.d/debian.sources /mnt/etc/apt/sources.d/.
cp /etc/resolv.conf /mnt/etc/.

echo "nas" > /mnt/etc/hostname
sed -i -e 's/localhost$/localhost\n127.0.0.1\tnas/' /mnt/etc/hosts

rootfs=$(blkid /dev/sde | grep TYPE=\"ext4\"|awk '{print $2}'|cut -d\" -f2)
vfat=$(blkid /dev/sde|grep TYPE=\"vfat\"|awk '{print $2}'|cut -d\" -f2)
swap=$(blkid /dev/sde|grep TYPE=\"swap\"|awk '{print $2}'|cut -d\" -f2)

cat >/mnt/etc/fstab <<EOF
UUID=$rootfs / ext4 noatime,errors=remount-ro 0 1
UUID=$vfat /boot/efi vfat noatime,umask=0077 0 1
UUID=$swap none swap sw 0 0
EOF

chroot /mnt
apt update
apt upgrade -y
apt install grub-efi-amd64 linux-image-amd64 ssh \
        firmware-misc-nonfree \
        firmware-realtek xfsprogs rsync pmount \
        gddrescue screen util-linux-extra bash-completion \
 	mdadm  parted smartmontools htop ntp unattended-upgrades sudo
useradd -m -G sudo -s /bin/bash -C 'Your Name' your_username 
passwd your_username
adduser your_username plugdev
apt install tzdata locales
dpkg-reconfigure locales
grub-install --target=x86_64-efi --force-extra-removable /dev/sde
update-initramfs -u
apt clean
exit # leave the chroot
umount /mnt/run
umount /mnt/sys/firmware/efi/efivars
umount /mnt/sys
umount /mnt/proc
umount /mnt/dev/pts
umount /mnt/dev

Note that required HDDs can be easily installed later. I manually configured the two disks with GNU parted for a GPT Linux RAID partition. After booting with the stick, a simple install of the md array support suffices. Typically, the USB stick runs as /dev/sdd

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mkfs.xfs /dev/md0
mkdir /data
data=$(blkid /dev/md0|grep TYPE=\"xfs\"|awk '{print $2}'|cut -d\" -f2)
echo "UUID=$data /data xfs defaults 1 1" >>/etc/fstab

In order to allow shutting down by pressing the power button, it is required to configure systemd-logind as follows.

sed -i -e 's/^#HandlePowerKey=poweroff'/HandlePowerKey=poweroff/ \
       -e 's/^#HandlePowerKeyLongPress=ignore/HandlePowerKeyLongPress=ignore/ \
    /etc/systemd/logind.conf

systemctl restart systemd-logind.service

It could also be a good idea to stop the periodic auto-scan on the RAID volume for big disks, which can take ages to run.

sed -i -e 's/^AUTOCHECK=true/AUTOCHECK=false/' /etc/default/mdadm
systemctl restart mdmonitor.service

The network configuration depends definitively on the type of connection used and the home network setup. In my case, the NAS uses a static IPv4 address, so it can be configured through ifupdown, and it is only necessary to correctly write the /etc/network/interfaces for the enp1s0 Realtek 2.5Gb Ethernet interface. Not that it requires a non-free firmware blob to run.

After the initial syncing, a series of software to better manage the NAS can be installed, but that is optional and can be the subject of a different post. For sure, for better convenience, a copy of the USB stick with the complete configuration is a good idea, to allow a fast recovery in case of failures.

A call to minimalistic programming

2025-09-10T17:00:00Z

Minimalism in development is a forgotten virtue of our time that should gain more attention. A straightforward summary of some minimalism principles is available here. Briefly, the principles of minimalism in Software Engineering can be summarized as follows, based on the manifesto for minimalism.

Fight for Pareto's law: look for the 20% of effort that will yield 80% of the results.
Prioritize: minimalism isn't about not doing things but about focusing first on the important.
The perfect is the enemy of the good: first do it, then do it right, then do it better.
Kill the baby: don't be afraid of starting all over again. Fail soon, learn fast.
Add value: continuously consider how you can support your team and enhance your position in that field or skill.
Basics, first: always follow top-down thinking, starting with the best practices of computer science.
Think differently: simple is more complicated than complex, which means you'll need to use your creativity.
Synthesis is the key to communication: we have to write code for humans, not machines.
Keep it plain: try to keep your designs with a few layers of indirection.
Clean kipple and redundancy: minimalism is all about removing distractions.

Most of those principles are coherent with each other and relate heavily to the well-known Unix KISS principle.

An extended and fascinating book about the practical application of such principles is Eric S. Raymond's "The Art of Unix Programming", which I strongly recommend reading. I can also recommend a now-classic volume on the same topic by John Ousterhout, "A Philosophy of Software Design". Both outline practical examples of how minimalism in design can be effectively embraced, with a focus on doing the right thing sooner rather than later.

The same principles could (or maybe should) be applied even to programming languages, but this is often a neglected aspect of such a minimalistic approach. Note that one of the most successful languages of all time is the C language, which indeed has a straightforward syntax and, as such, cannot be easy to use correctly (the principle is that what is simple is not necessarily easy, too). That's because the programmer needs to create her/his own abstractions and layers to build her/his vision of a software design. It seems that this is precisely the opposite of the C++ or Java approach, where the entire specification spans thousands of pages, and many high-level abstractions are integral parts of the language. The same can be applied to Python nowadays, which started as a simple language, more readable and clean than Perl, but now has a wide and articulated specification. Again, hundreds of pages are now needed to describe a once-simple language, where tons of new features and abstractions have been added to enrich its expressiveness. If one considers its standard libraries and modules, the actual situation appears even worse. Can such an approach be considered easier? I don't think so. Let me say: how can a program be considered simple if it relies on hundreds (or even thousands, including dependencies recursively) of external modules, as well as hundreds of syntactical constructs and glues? Some languages also manage multi-versioned dependencies, allowing a program to cross-depend on multiple editions of the same module (yes, JavaScript, I'm talking about you), with the concrete possibility of introducing obscure bugs as a result. At the opposite extreme, there is the consideration that we only know and deeply understand what we make.

Minimalism also means actively seeking a balance between these two opposing approaches, because reusing third-party modules and packages can be an immediate solution to deadline urgencies, but can also potentially introduce instability and dependencies on unmaintained software in the long run. Long dependency chains where changes happen independently of the main program focus and are introduced by third-party motivations and reasons - often with wrong timing for depending projects - can cause breakages at multiple levels.

Of course, to reach the right tradeoff, a few things need to be considered: every single programmer could not be smarter than a lot of libraries and modules out there, where multiple developers could have spent hours/weeks/months, or even years refining them. That's true, but it is also true that not all libraries or modules are written with the same level of quality and effort. For instance, we all know cases of elementary modules available for Node that could be easily avoided, and instead are imported for some form of laziness in development. Even, sometimes features that need to be used could be only a small portion of the whole library/module, which could be reimplemented with a very reasonable effort and time. This approach could be amplified in modern times when AI tools could significantly increase productivity in such cases. I would simplify these concepts with some additional mottos:

Limit your external dependencies: avoid depending on modules or libraries that are not strictly required to significantly reduce the total development time, are not rock stable for their interfaces and features, and do not have a clear and stabilized roadmap.
Reproducibility of the software stack is a must: these days, a SBOM has become recommended/mandatory, but it should not only consist in a documentation of external dependencies and their versions, but also the full process of building a runtime environment should be fully defined and consistent for the long term.
Do not follow the last oh!-so-cool technology: while that could be done for an amateur project to develop during spare time, it is not a good idea depending on a technology whose future is not clearly stated, with a well-established development team and proven sustainability in the long term. I consider a risk even depending on a single company project, and even more if it is considered a startup. Synthetically, this can be generically considered as minimalism in coding style.

Moreover, if you are going to use a well-established framework, such as Django, for developing your mid-to-long-term web project, it is probably better than using the latest Nodejs-based framework created six months ago that seems the latest 'big thing'. But that's probably only common sense. Instead, ask yourself if your project should be created from scratch using a simple jamstack system and some microservices for well-defined and minimal parts. In many cases, that is more than enough for too many CMS-based sites out there: indeed, I continuously ask myself why a lot of websites are still based on WordPress, when most of them could be easily converted into a handful of static pages and simple JavaScript snippets that they will use in any case. This can be declined in terms of minimalism in defining computing architectures, which can also allow scaling up applications more easily.

So minimalism principles can be considered at multiple levels: for programming languages, libraries, architectures, and design. However, they require skills, in-depth research, and a significant amount of time to dedicate to continuous refactoring and meditation about viable alternatives. And that's probably the key point: developers with deadlines and urgency imposed by PMs are too often tempted to follow the easiest and richest paths and provide a solution of any kind without too much meditation on the final balance among efforts, quality, efficiency, and durability of results.

Of course, about minimalism, an extraordinary citation is due for the whole suckless effort on the uncompromising minimalism side. And why not?.

Ok, ok, I'm joking. But you got the point.

Does HPC mean High-Pain Computing?

2025-09-06T19:40:00Z

Please, forgive the silly joke in the title of this semi-serious post, but lately I have been thinking about the strange fate of an area of general computing that I have spent more and more time in recently, as in the near and far past. For my job, I have utilized a series of scientific HPC clusters worldwide to solve multiple computing problems most efficiently by distributing computation across numerous nodes. Over the last thirty years, all such platforms have consistently shared the same common characteristics, which invariably pose a problem in their use for the average scientist (often a young/junior dedicated to a short-term project) in any application domain.

To use Fred Brooks' definition, HPC technologies have both intrinsic and incidental fallacies for such users category. The intrinsic one is due to the inner complexity of creating a parallel and distributed solution to any problem, possibly in a way that does not harm the final implementation due to the increase in communication time among computational agents. This is already a relevant problem per se, which can often be out of the abilities, knowledge, and interests of the average researcher in bioinformatics, physics, mathematics, remote sensing, or whatever other research domain.

The incidental fallacy is instead always due to the accessibility of platforms and the technologies used for their implementation. At large, all such HPC clusters are a large pool of multi-core hosts with plenty of memory and connected with multiple high-speed networks for implementing some sort of multi-tier distributed POSIX file system and/or object storage. Users can log in on a limited number of such hosts that are connected to all others and run some type of scheduling system (e.g., Slurm or HTcondor) where multiple computational nodes can be reserved for a limited period of time to execute batch jobs or even an interactive one (mainly for debugging). In most cases, such clusters can also be used with some MPI/OpenMP implementations for proper parallel computational modeling based on message passing among computing agents that run on multiple cores and hosts, with or without multi-threading. Alternatively, GPUs can also be reserved and exploited via Cuda/OpenCL. In many cases, such implementations are vendor-oriented and trigger the need to adopt specific libraries and compilers that add another layer of complexity to implementations.

The incidental problems start when the casual users discover that all such computing nodes invariably run some legacy enterprise Linux distribution that is maintained for a period of ten years or even more, until a full reinstallation of the whole cluster. On top of such legacy systems (that are for any practical use simply unusable as such) these scientific clusters give essentially a few different mechanisms for creating a general computational environment:

Environment Modules
Containers (Singularity or Apptainer)
Anaconda/Miniconda-like environment (or free forks like Miniforge)
Some specific software/application to run

But for containers, the other solutions are all binary-based hubs, which could expose them to possible breakages when the application developed needs to access exotic language bindings for extensions, and the poor users enter the mysterious and dangerous world of ABI violations and a chain of broken dependencies. Even, often, such hubs are not always consistent, and any upgrade by the admin team exposes them to sudden breakages from night to day.

The final solution (or apparently so) nowadays is using containerization and a target environment where the user code can find all and only the correct dependencies and versions for the whole software stack of the application. This, at least, until the third-party hubs of base distributions and languages ensure complete consistency and retain past binaries and versions for any medium/long-term need. Of course, a full source-based stack with proper version tracking a la Guix would help to avoid dependencies on external binary hubs and seems the way to go. Indeed, a small group of interest in such a solution has existed for a few years, but I am unaware of so many HPC clusters that consistently propose this kind of implementation for users. That said, writing Guile Scheme descriptors for preparing an execution environment may not be within the reach of the average researcher in biochemistry or astrophysics.

Unfortunately, as I wrote in a past post on this digital site, this moves the whole responsibility of a software stack maintenance onto the shoulders of the final users, who are often the infamous junior profiles I mentioned before. These are non-IT specialists who should adopt such HPC platform to implement solutions as part of their daily job in their special scientific domain.

The result, to be honest, is that the average researcher simply tries to avoid the whole thing as soon as possible because of the significant complexity that the entire thing involves, while the private sector introduced specialistic roles of data and software engineers to manage such problems properly (which is the only reasonable approach, indeed). Adding insult to injury, in some academic areas, such interests in HPC are also viewed with contempt or as a waste of time, if not openly discouraged.

All this explains why a roundabout in any of the significant HPC clusters worldwide often guarantees hilarious experiences in terms of who is doing what and how.

Sometimes, I almost feel like I can hear them swearing...

Guix for geeks

2025-09-03T19:40:00Z

In the last few months, I have installed and upgraded my second preferred GNU/Linux system, GNU Guix, on multiple boxes. Regarding that system, I have already written a few introductory posts in the recent past. This is an update about my experiences as a user and developer. I still think Guix is a giant step forward in packaging and management, in comparison with Debian and other distributions, for elegance and inner coherence.

On the negative side, I can confirm that the most important aspects to consider in order to adopt Guix for daily use are as follows.

Guix is essentially a roll-on type of distribution with a limited user base, so a grain of salt is required at upgrade time because instabilities are somehow expected. The integrated time machine can be used to step back in case of problems, anyway.
The development team and other support teams are tiny enough that I would avoid using Guix for publicly exposed services. Also, it can lag on specific applications and issues. Tasks and goals do not specifically structure teams, so there is no proper security team, but I guess all maintainers can participate to cover all required tasks.
Packages are not always up-to-date with the latest upstream versions. Some packages are even older than in Debian stable, so if you are looking for the latest and coolest products, that's not the place.
Some choices for the core distribution are not mainstream, such as using shepherd as its init system instead of systemd. This implies some delays because patches are sometimes due for complex software, such as Gnome, which introduced strict dependencies on systemd in recent years. This is the reason why, as of the current date, Gnome is still at version 46, while Debian stable is at version 48.
The geospatial tools and libraries available are still quite limited, which may require a more in-depth analysis to estimate the level of lag. See the section below.
The readiness for immediate use is not comparable to the main distributions. For instance, if one absolutely needs a non-free Linux kernel due to firmware constraints, the nonguix add-on repository is available; however, one must build and install an ad hoc ISO installer by hand. Nothing transcendental, but complicated enough to discourage most users. Here is my fork of the System Crafters repository to prepare such an ISO image, updated for use of the current main Guix (now hosted on Codeberg) and NonGuix channels. That could be easily prepared on any foreign distribution that includes the guix package management software, such as Debian.

As of the current date, I have installed at least a few laptops and VMs, including a Dell Inspiron 15, an Asus EEE 1215P, and an old Acer Travelmate. I strongly suggest using at least a box with 8GB of RAM and a 512GB SSD, because building from sources can be overwhelming. My dual-core Atom EEE Pc has only 2 GB of RAM, and it is slow enough, but it has the big advantage of being fully supported by the official GNU Linux libre kernel.

Additionally, I believe that having a local build host for substitutions within the home/work LAN is the most efficient solution. That is because the officially provided worldwide substitution hosts could be generally heavily loaded and occasionally fail (when that happens, the time of recovery is on a best effort basis).

Of course, it is also entirely possible to use Guix only as a package management system on a foreign distribution, having most of the advantages of a reproducible environment, as well as within a container or a virtual machine. In that case, one has to consider that the host system is typically based on the systemd init system, which can cause a few headaches when porting existing package descriptions for services taken from the Guix repository. Probably the best compromise is still using a foreign distribution on the physical box and a Guix-based container to prepare an execution environment, something I already experimented as explained in previous posts.

Guix and geospatial software

In 2004, Paolo Cavallini and I started a subproject within the Debian community of developers and users to improve the status of geospatial software in Debian. That was the birth of what is today the DebianGIS pure blend, with a mildly coordinated team of developers and maintainers that work together in Debian (and derivative distributions) on a set of essential libraries and tools for the geospatial community. For people interested in the history of FOSS, I wrote a contribution to the ASITA conference in 2009 about that. That was and still is grunt work, often neglected and at the time perceived as a secondary effort by upstream developers. Indeed, that is an essential task instead, because collecting together properly and ensuring that you have a complete and well-built stack of software for a geospatial application is not trivial at all.

One of such base libraries is GDAL, which can be built with multiple optional dependencies and plugins. Like other platforms, some of those optional dependencies are missing in the Guix package, and that should be taken into consideration. Some tools are instead totally missing, such as MapServer and others. Of course, some packages could be patched here and there in Debian, or can require a Guix-specific patching (because of the peculiarities of such a system, which does not respect the Linux FHS), and that definitely could be a reason to have a package in good shape or not. That's to say that the Guix ecosystem would need a serious help for packaging such niche packages, but where most people can see a lack, I see an opportunity in that regard. After all, when I started my life in the Debian project, it was because I did not trust an operating environment that I could not develop and adjust personally for whatever reason. That's the way and Guix seems a brilliant example of a vibrant community for that.