Crosswords for GNOME

Jonathan Blandford, who is a longtime GNOME contributor—and a cruciverbalist for longer still—thought it was time for GNOME to have a crossword puzzle application. So he set out to create one, which turned into something of a yak-shaving exercise, but also, ultimately, into Crosswords. Blandford came to GUADEC 2022 to give a talk describing his journey bringing this brain exerciser (and productivity bane) to the GNOME desktop.

Blandford got his start with GNOME back in 1997; he is the author of the Evince PDF reader and Aisleriot solitaire card game, for example. He has moved into management over the years, so has done less programming for GNOME, but is still involved in the community. For the purposes of the talk, he said, the important thing to know about him is that he is "a passionate cruciverbalist". He has been doing crossword puzzles since he was a child, with his parents and grandparents; now he does them with his family as well. Beyond that, he had to work the word "cruciverbalist" into his talk because "people who love crosswords love words, and they love words like that".

He started working on the Crosswords application in mid-July 2021—almost exactly a year before the talk. It is now around 24,000 lines of code, most of which is in C; "I have some regrets about that" but he was comfortable writing in C. The program has been translated into two languages beyond English—Dutch and Spanish—and the project has three additional contributors at this point.

There were several motivations that led him to create the application. He has always thought that puzzles would make for an interesting program to write. In addition, anyone searching GNOME Software or Flathub for a crossword puzzle program would not have found one; in fact, Linux, in general, lacks for a good graphical crossword application. Writing a program for these puzzles turned out to be "a whole lot of fun" as well. But he wants its users to also experience fun; he had an overarching rule for the program: "it's a game, it has to be fun".

Crossword 101

There are a bunch of different kinds of crossword puzzles, starting with the first, which was by Arthur Wynne in 1913; the Crosswords application supports a variant of that type of puzzle. Currently, some of the crossword types are supported by the game and he would like add others; "the one thing they all have in common is that they are all boxes with letters in them".

The two most popular types are the "standard" (or "American") style (seen at left from Wikipedia), which has a grid that is more dense. It uses "straight" clues that are often constructed around a definition for the solution word(s), perhaps with some slight misdirection. Sometimes there are multiple possible solutions for a given word, so it may be necessary to solve some intersecting words to narrow the solution down. That type of puzzle is common in the US as the name might imply.

The second is the less-dense "cryptic" crossword (seen at right from Wikipedia) that is more common in the UK, South Africa, and elsewhere, though it is becoming more popular in the US as well. Because the grid is more open, there are fewer opportunities to gain help from intersecting words; once the right answer is found, though, there is generally no question whether it is correct. The clues have two parts, a definition that normally appears first in the clue, followed by some word play that is used to build the answer.

He went through an example from "16 across" in the "London Times Sunday Cryptic No 4962" by Robert Price, which was: "Standard heroes possess grit alongside guts (5,3,7)". The numbers indicate the lengths of the words making up the answer—"stars and stripes"—which is a "standard", in the sense of a flag. Heroes are also "stars", who "possess grit" because "sand" is found inside "stars" (i.e. "starsands"). Meanwhile, "starsands" is alongside "tripes", which are intestines, thus guts. This word play acts as a sort of checksum to validate a possible answer, at least in his engineer's brain.

Crossword puzzles tend to be specific to a particular region or time period, which is part of why they are so closely associated with newspapers. It is difficult to create a puzzle that will cross cultural boundaries because the clues tend to refer to recent events, popular culture, and local vernacular. Writing a generic puzzle that avoids those "localisms" would be extremely difficult, he said.

Getting started

The first thing he needed to figure out was what file format to use. He thought about creating his own, but also did a survey of the existing options: .puz, .jpz, and .ipuz. The .puz format is proprietary but is by far the most popular; there are a lot of puzzles available in it. It has been reverse-engineered, but it has limited capabilities and is not extensible. .jpz is XML-based, so it is easily parsed, and there are a good number of puzzles available in the format, but it also has limited capabilities.

The .ipuz format is well-documented and explicitly free to use. But it has seen limited use in the wild, so there are not a lot of puzzles out there in .ipuz format. It supports a lot of functionality so it can handle lots of puzzle types beyond just crosswords, including Sudoku, sliding-block, trivia-quiz, and word-search puzzles. It also has lots of options for styling puzzles, which allows for custom shapes and colors for themed puzzles.

But .ipuz is JSON-based, and "it's very webby, which is one of the things I don't like about it". The format kind of assumes that the program will load the JSON structure and work with that directly; the raw structure is not exactly what he wanted for Crosswords, so he created libipuz to load and save the format. It does not mirror the JSON structure directly, though it is inspired by it; his intent is that others could eventually load other puzzle formats into the format used by libipuz.

In about five months, he had something working to where he could display puzzles and their clues that were loaded from .ipuz files. The answers could be typed into the boxes and the results could be saved. He started out with a traditional callback structure, but Federico Mena Quintero introduced him to data-oriented programming with a unidirectional data flow, which was a better way to structure things.

A PlayState structure was used to hold the complete state of the game at any given point in time; no game state is stored in the widgets. PlayState is immutable, so changes to the game state result in a new PlayState. When an update is made, all of the widgets will update how they are displayed based on the state of the game at that point. Doing things that way made it much easier to test the application and to reason about how it worked.

At that point, the game would just load a puzzle from a tiny library, allow the user to play it, and then save it back to disk; he asked, is it fun yet? The answer is "not really". The lack of puzzles is a real problem, so he set out to write some.

More tools

It turns out that creating crosswords is surprisingly difficult. Creating a good board for standard crosswords is hard—there are lengthy essays about it from the pre-computer days—while the cryptic boards are easier, their clues are not. It takes a lot of practice to create cryptic clues and in doing so he realized that they tend to be highly regionalized.

Beyond that, Emacs is not the best tool for writing crossword puzzles, either; there is no crossword mode, though perhaps there should be. In any case, he was able to create a few puzzles, the first of which was in a 5x5 grid with ten clues; he manually created the JSON in Emacs and was able to load it into the Crosswords application.

He realized that his technique was not going to scale, so he set the game aside to work on a crossword editor. He looked at other crossword-editing tools and found that there were no good free options, though there are a few proprietary options, each with its own quirks. The basic workflow of his editor, which is accessed as an option in the Crosswords application, is to choose a grid type and size, then to fill in the solution on that grid.

He quickly got to the point where users could fill in the grid, but discovered that actually doing so is "surprisingly hard". He spent a lot of time looking up words as he was trying to fill in a grid, trying to find words that would fit into the space provided. He discovered that he needed a crossword solver to help crossword writers fill in their grids.

He wanted to be able to show a list of words that would fit based on the spaces available and the letters already filled in. He used Peter Broda's crossword word list and built a mechanism to filter the list based on patterns. So "?NO?E" would produce a list of words like "SNORE" and "SNOKE"; there are about 12 other words that fit the pattern from the list, he said, including, of course, "GNOME".

He has an implementation of the filtering that is "pretty fast"; most queries take around a microsecond. He created a table that can be loaded into memory with mmap(), which contains the words themselves and a set of indexes for "fragments", lists of words matching a single letter in a particular spot. So, "?NO?E" is broken down into three fragments, "?N???", "??O??", and "????E", each of which is looked up in the fragment table and any word index that appears in all three fragment lists is a solution. He hopes to use that data structure for finding anagrams eventually as well.

Getting suggestions in the editor was helpful, but it was still a laborious process to create the puzzles. Sometimes he would choose a word that eventually led to corner that he could not complete, but he might end up wasting an hour before finding that out. So he took the next step and used the word list to create a solver that can suggest words that all fit together into part of a puzzle.

The solver is "a little buggy but it basically works"; a user can select an area of the board and the solver will try to find words that can fill in the region. It does up to about 30,000 boards per second, depending on the size of the region and the lengths of the missing words, though there is lots of room for improvement in the performance, he said. It can also help find areas that are not going to be easily filled, or can only be filled with words the creator does not want to use, so that the puzzle can be reworked to avoid that.

There is a lot of trial and error in creating a puzzle, so he added a way to undo and redo changes. The feature was also incorporated into the Crosswords game, but it came about because he needed it in the editor. For example, changing a letter box to be a block redoes the numbering throughout the puzzle, which means that the clues get changed (or dropped), and so on. Since the blocks in puzzles are traditionally symmetrical (either bi-lateral or rotational symmetry), adding a block might automatically add it elsewhere in the puzzle, further rippling the destructive effects.

Since a minor change (or even misclick) can cause massive changes, undo and redo were particularly important—and not that hard to implement as it turned out. Since the PuzzleState holds everything about the puzzle itself, it just needs to be augmented with the current values of the widgets (e.g. letters for words that have been added). He then has a list of these entities that the program can move through in both directions; at each point it instantiates the state of the puzzle at that point in time.

Cats and dogs

He was able to use the editor to create his first "Cats and Dogs" puzzle set, which is a collection of nine puzzles, some in non-traditional shapes, that have a common theme (overview seen at right). He encouraged people to go to Flathub and install Crosswords to decide "if it is fun or not". He has some other themes that he plans to do puzzle sets around in the future.

In order for users to access puzzle sets, however, there needs to be a way to package and distribute them. He added the concept of puzzle sets to the game, but he wanted the sets to be standalone files that could be distributed on their own. The puzzle sets are stored in a GResource file and can be distributed as Flatpak add-ons. There are also mechanisms to download puzzles from some web sites, and he would like to add more sites as time allows.

He took a visual tour of the Crosswords application using his slides, rather than give the dreaded live demo. The YouTube video of the talk and his introductory blog post will help with visualizing the interface. Blandford did reiterate his suggestion that people try it out from Flathub as well. The application has ways to choose puzzles or puzzle sets, as might be expected. Puzzle sets can have some puzzles locked depending on which other puzzles have been successfully solved.

The heart of the application is the puzzle-solving interface (seen at left). It has the ability to tell you when you have made a mistake (the red letters) and it can make suggestions using the word list that he added for the editor. "Is it fun yet?" He said that it was starting to get there, but there is a need for a lot more puzzles. Overall, "I am pretty pleased with how it looks today", he said.

Blandford does, of course, have plans for the future, starting with his work on a second puzzle set. He hopes to see the development team expand and for more features to be added. Support for barred crosswords has just started; Acrostics support would be a great addition as would support for printing puzzles. There is a lot of work to be done for internationalization, including support multi-byte character sets in the word list. Supporting puzzles for other languages is "way more than just translations" as there are various language-specific quirks that need to be handled.

He would also like to grow a puzzle community. He hopes to see a community of puzzle writers using the editor form and create free-to-distribute puzzles for others to enjoy. There are features that he wants to see get added to the editor, as well, in order to provide more help on the clue-writing side. Perhaps a crosswords.gnome.org site could come about with competitions and more. He encouraged those interested in any aspect of the project to get involved.

[I would like to thank LWN subscribers for supporting my trip to Guadalajara, Mexico for GUADEC.]