Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The method we use is a variation of the Agile method called Scrum. This method takes its name from the team’s daily meetings, which are compared to a rugby scrum.

How does it work?

In practice, the project is broken down into a set of tasks (called stories) and these stories are spread over recurring time periods (called “ Sprints”).

The idea is to involve project clients at regular intervals by working iteratively.

A Sprint passes, the client sees the completed work and decides whether to make changes or continue the project, another Sprint is then formed. And so on.

An Agile team usually has fewer than 7 people. Beyond that, the team loses efficiency.

In this series of articles, you will see how to formalize stories to meet the specific needs of the project and the client, and how to structure your Sprints to keep them effective.

The Agile method is often associated with DevOps techniques (and DevSecOps, MarketOps…) which we will cover in a future series.

Where do we start?

Well, let’s start at the beginning of a project, the Discovery & Framing phase.

Then, I invite you to read the articles on stories, the Ceremony, the Backlog, Scrum and Sprint to get an overview of this method.

After that, feel free to explore. The hyperlinks are there for that.

The ceremony is the meeting that brings together the agile team (the development team), the Scrum Master (the “manager” of the team), and the Product Owner (the client representative), and occasionally the clients.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

It usually takes place between sprints, at regular intervals.

This event is divided into several phases. Some teams choose to separate these phases to lighten the meetings.

Phase 1, at the start of the project (Sprint Planning Meeting)

Ceremony Master: The Product Owner assisted by the Scrum Master

Duration: less than 2 hours.

This phase allows detailing most of the stories in the form of Stories, NFR, or Spike and distributing them across the planned sprints.

It also allows estimating the broad outlines of the project. It focuses on team cohesion and getting the team familiar with the clients and the project.

The goal is to start the first Sprint (Sprint 1 or Sprint 0) under good conditions.

Phase 1, during the project (Sprint Review)

Ceremony Master: The Product Owner assisted by the Scrum Master

Duration: less than 1 hour.

This phase allows reviewing the previous Sprint. During this phase, the agile team presents the results of the Sprint to all project stakeholders. Demonstrations may be made during this phase to clients, end users, or a representative panel…

Clients, through the Product Owner, should get an idea of the project’s current state.

During this step, all participants provide feedback to the agile team. This is where the project can be reoriented and phase 2 prepared.

Little Limawi bonus, between the two phases

Ceremony Master: None, it’s a card game

This inter-phase consists of reviewing threat models using the “ Elevation of Privilege” game. This method is explained in a dedicated article.

Phase 2, project-focused (Product Backlog Refinement)

Ceremony Master: The Product Owner assisted by the Scrum Master

Duration: less than 1 hour, can be very quick.

This phase allows reorganizing the stories ( Stories, NFR, Spike) for the next Sprint. The agile team questions the Product Owner about the stories to make them as clear as possible. The team then estimates the workload of all these stories (in terms of complexity points or time) using Agile Poker. Once the stories are estimated, the Product Owner prioritizes them according to the team’s velocity. He/she places them in “TODO” and sends any excess back to the Backlog (marked as “POSTPONED”).

This phase can occur in small touches throughout the Sprint. In this case, the Product Owner can refine much better and faster according to the challenges faced by the team. If this approach is preferred, the phase within the ceremony can be very short.

Phase 2, team-focused (Sprint Retrospective)

Ceremony Master: The Scrum Master

Duration: less than 1 hour.

This phase focuses on the team’s health. It can be done in the form of games. It helps strengthen team cohesion and continuous improvement.

The goal of this phase is to re-motivate the team, address social relationship difficulties among team members, and improve the work environment.

There are several types of stories, but they all share having a goal, being estimable (or breakable into estimable elements), and having a clear definition of “DONE” (a clear end).

These units allow a project to be broken down clearly so it can be carried out as efficiently as possible.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Here is a (non-exhaustive) overview of the different types of stories

• Task: Not really a story per se, but it is the atomic unit on which a story is built. It should be as simple as possible. For example, if the story is “entering a house,” the task is “place the hand on the door handle,” the next task “turn the handle,” etc.
• Story: This is the most well-known type of story in agile methodology. It is focused on interaction with the end user. To avoid confusing “histoire” and “story” in English, the term “User Story” is preferred. It allows the realization of a feature.
• NFR: This type of story is the hardest to define. NFR stands for “Non-functional requirements.” These are all actions necessary for the smooth running of the project but do not affect the end user’s interaction with the project. For example, code refactoring, a license change, setting up a new development server, or a coffee machine for the team.
• Spike: This story formalizes research steps. It has a different definition of DONE than others because research inherently involves uncertainty.
• Epic: This definition varies depending on who uses it. Generally, it defines a story whose estimation exceeds the capacity of a Sprint. It must therefore be broken down into smaller stories.
• Initiative: Used by JIRA. It groups together a set of Epics linked by a common goal.
• Theme: This story groups Epics or Initiatives whose completion represents a major change for a large part of the organization (or company).
• Saga: This term is rarely used (but I like it) and can be confused with Theme or Initiative. It gives a sense of continuity akin to “epic.” It can define the root of the project, the story tree. Therefore, I suggest calling the story tree “Yggdrasil” to remain consistent in terminology.

Story Start and End

Each story must have a clear definition of READY (or TODO) and, most often, DONE.

That is, the Product Owner and the agile team must agree on the prerequisites for starting the story (READY). For DONE, they must define the criteria that ensure a story is truly finished. Note that stories like Epic, Theme, Initiative, or Saga do not have a DONE definition other than the DONE of all the stories they comprise.

Some Stories in Detail

User Story

The most well-known story in agile methodology. This type of story creates functional value for the end user.

Definition of READY: A story must have a clear start according to the INVEST acronym below:

• Independent: Each story must be independent of any other task.
• Negotiable: Each story should be discussable to best describe the user journey it provides:
  • Given a state
  • When an action
  • Then a result
• Valuable: Each story must deliver actual value to the product from the perspective of the end user.
• Estimable: Each story must be estimable in terms of complexity points, leading to a time estimate.
• Small: Each story should be small enough to be completed by one or a few people in a Sprint.
• Testable: Each story should be testable within a continuous integration testing tool.

Definition of DONE: A story must have a clear end defined in advance with specific objectives.

Example of DONE:

• Generated documentation
• Functional tests passed
• Security tests passed

NFR

NFR is an agile story type that does not focus on the end user.

NFR stands for “Non-functional requirements.”

It addresses tasks necessary for the smooth running of the project but invisible to the end user.

Examples:

• Legal issues
• Security issues
• Organizational issues
• Performance issues
• Infrastructure issues

An NFR is similar to a Story, except it is not focused on the end user.

An NFR must be SMART:

• Specific: understandable by everyone
• Measurable: with a clearly defined DONE (objective)
• Achievable: feasible
• Relevant: pertinent
• Time-Boxed: short-term and estimable in complexity points

It must have a clear definition of READY and DONE.

Spike

Used for research in agile methodology.

A Spike (the tip) aims to clarify steps, knowledge, and time needed to resolve Stories and NFRs.

It must be independent and testable, but by nature, it is not estimable. Indeed, in research, the outcome is unknown, making estimation impossible.

To address this, it is time-boxed (with a defined time).

Example: Researching a CouchDB replacement in 24 hours (3 days)

A Spike must be SMART:

• Specific: understandable by everyone
• Measurable: with a clearly defined DONE (objective)
• Achievable: feasible
• Relevant: pertinent
• Time-Boxed: short-term

It must have a clear definition of READY and DONE.

The DONE in this case defines the goal and research direction. It helps determine whether the research is valid.

For example, if the goal is to find a new method for making a raspberry pie and the research results in methods for making cat kibble, it is not DONE.

Sources

• Scrum Expert
• Atlassian

Agile Poker is presented in the form of a set of cards, with each person receiving a complete deck. During the Sprint ceremony, the team uses the cards to evaluate the complexity of the stories ( User Stories, NFR, or Spike) being considered.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Complexity Point

To estimate the difficulty of completing a story, the first instinct might be to determine a duration.

This approach can create a stressful environment. Indeed, a developer (or marketer, etc.) responsible for the story might feel pressured by time and complete the story too quickly and poorly.

Of course, the review phase can address this issue by repeating parts of the story over and over, but this is a waste of time, energy, and resources.

This is where the complexity point comes in.

This tool allows assigning a complexity value to the story. The value can be relative to other stories or absolute.

In software development, the agile complexity point can mirror the complexity of the code to be written. For example:

• one class = 1 point, 3 classes = 3 points
• 1 point per method
• based on calculated complexity (e.g., in PHP)

Each story is assigned complexity points via Agile Poker.

Velocity

The complexity point is not initially tied to duration.

Once an agile team becomes well-practiced (i.e., after several Sprints), it is possible to determine the average number of complexity points the team can complete per Sprint.

This is velocity.

It is a batch approach, but trying to over-detail the average time per complexity point can create stress again.

Once an average velocity is estimated, the goal is to measure the team’s performance and maintain that velocity.

It is not always wise to try to increase it, to avoid reverting to a stressful situation.

Poker Rules

For each story, every team member selects a card from their deck and keeps it hidden. The card represents their estimation of the [story]’s complexity.

Once everyone has chosen, all cards are revealed simultaneously.

Those whose cards are farthest from the average explain their reasoning. A consensus is then reached on the team’s estimated complexity.

The available decks below are 7 in number (for a standard agile team) and are distinguishable by color.

Each deck contains:

• a Fibonacci sequence: for precise estimates or workdays (for Spikes) using the grid in the center (one column represents a full week);
• T-shirt sizes: for less precise estimates;
• a plus and minus: to approve or disapprove;
• a question mark: when unsure;
• a coffee: when it’s time for a break;
• infinity: when the story is too large and needs to be broken down.

Sources

• Atlassian

The Backlog gathers all the stories of a project. This list is usually digital but can also be displayed on post-its.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

This list is maintained by the Product Owner with the help of the Scrum Master.

Each story, within the Backlog, has a label describing its type ( NFR, Story, Spike…) and progress or scope labels. All project stakeholders can add stories to the Backlog, as it gathers all sources of project actions (Bug Tracker, User Feedback, continuous integration feedback, logs, error reports, discussions between stakeholders…).

At each ceremony, the Product Owner extracts a certain number of stories from the Backlog and assigns them a progress label. These extracted stories form a sub-Backlog called the Sprint Backlog. The main Backlog, in this case, is called the Product Backlog. Note that Backlogs can be hierarchized as needed to optimize clarity and maintain efficiency.

All stories in the Sprint Backlog must be estimated by the team using Agile Poker.

The scope label can be defined by any project stakeholder following the project management strategy (team members or only the Scrum Master, for example). It also defines a sub-Backlog and allows multiple teams to work in parallel or directs stories to the team members best suited to handle them.

Examples of Labels

Progress labels I use:

• DISCUSSION: This story has not been analyzed yet. It’s a new input, a new idea…
• TODO: This story has been estimated and extracted from the Backlog by the Product Owner for the current Sprint Backlog. It has not yet been started by the team.
• DOING: After TODO, once the team has started working on the story.
• REVIEW: After DOING, once the team believes the story is completed. A team member who did not participate in this story or an external reviewer evaluates whether the work matches the story. There is often a back-and-forth between DOING and REVIEW to refine the match between the story and its implementation.
• DONE: After REVIEW, once the reviewer has given the green light. The story is completed. The definition of DONE is important and may vary depending on the type of story and the initial definition given during Sprint 0 or Discovery & Framing. Typically, the Product Owner moves the story to DONE.
• POSTPONED: This story was misestimated or a problem occurred during the Sprint. It is postponed to a following Sprint.

Scope labels:

• SECURITY: This story involves security aspects. It is handled by specialized security members.
• PERFORMANCE: This story involves performance aspects. It is handled by specialized performance members.
• DOC: This story involves documentation. It is handled by specialized documentation members.
• TEST: This story involves testing. It is handled by specialized testing members.

Scrum Board

The stories from the Sprint Backlog can be displayed on a Scrum Board.

It is a column-based board, with each column representing a progress label. The columns are arranged left to right according to the logical order of progress labels.

The idea is to move stories from left (TODO) to right (DONE). An effective Agile team has all stories in DONE by the end of the Sprint.

“Discovery and Framing” are two associated terms (discovery and framing) that describe an initial phase preceding the sprints.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

This phase is used to rough out a project and formalize it so it can be conducted using the agile method. It generally spans one week.

It is structured in two parts: discovery and framing (hence the name).

Discovery

It starts on Monday and ideally ends on Wednesday.

Each day, hypotheses about the project are detailed and clients are interviewed extensively. A meeting is held every morning to gather results and start again (similar to scrum).

Do not try to estimate time; this is a research phase. If this phase extends too far beyond Wednesday, it may be necessary to schedule another week.

Framing (Formalization)

From Wednesday onwards, you can start creating workshops within the teams. These workshops aim to define the main identifiers of the project:

• The “buyers personas” and final users personas: this technique is often used in marketing to target markets. In this context, it should be approached more broadly and technically. How will end users interact with the system? What are their cultures and backgrounds to anticipate the system’s usability?

• Determine the minimum viable objective to achieve. That is, the set of features absolutely necessary for the first version of the system. This is the “Minimum Viable Product” (MVP).

• From this data, create a Lean Canvas that will serve as the overall project sheet.

• Determine the necessary development environment and tools to develop, test, and deploy this first version.

• Begin detailing the epics, sagas, themes, and initiatives of the project, but not yet the stories, NFR, and Spike. The goal is to have a long-term vision of the project.

Wrap-up and Next Steps

This week concludes with a retrospective meeting (recalling the ceremony) to review progress. Of course, this meeting should not exceed 2 hours to remain effective.

This Discovery & Framing phase can be repeated (iterated) over several weeks until all project participants are satisfied with the main identifiers.

Once this phase is complete, the Sprints begin. Depending on the project, a Sprint 0 can be scheduled at this point, serving as a warm-up Sprint.

Threat Modeling

A threat model is a tool that helps identify a system’s vulnerabilities in advance and determine ways to address them.

In agile methodology, it is continuously updated. It must follow the existing product and quickly adapt to new developments.

How to achieve this?

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Diagrams and Schematics

First, it is necessary to document your project. It should have visual representations of its components and information flows.

One way to represent this data is using UML. The problem is that this language is complex to write and update manually.

It is therefore preferable to automate documentation and diagrams as much as possible (whether in UML or another format) to produce them easily with each project update.

If diagrams are done manually, simple sketches are enough. They just need to be understandable by all team members.

Once these diagrams are done, they can be analyzed to identify vulnerabilities—playfully, using the game “Elevation of Privilege”.

Game Rules

For this, plan a phase of the sprint ceremony dedicated to it.

The team sits around a table. The diagram of the project section to be analyzed is spread out on the table for everyone to see.

The “Elevation of Privilege” card game is available below.

Deal all cards to the players. The game starts with the “3 of tampering”. Play proceeds clockwise.

It is very similar to the rules of Tarot.

Each player continues in the same suit if they have a card in that suit. Otherwise, they play a card from another suit.

Each trick (one round) is won by the player with the highest card in the lead suit, unless an “Elevation of Privilege” card is played (in which case the highest of these cards wins).

To play a card, the player must announce it and try to find its corresponding threat on the diagram. The system may be resistant to this threat. In this case, the threat cannot be found on the diagram.

The player must clearly state the threat. For it to be valid, it must lead to the creation of a story ( opening NFR, User Story…) in the project.

At the end of the trick (when all players have played a card from their hand), all players who successfully identified their threat on the diagram score a point. If the trick winner also identified their threat, they score an additional point.

The trick winner starts the next trick and chooses the lead suit. Take a few minutes between tricks to review threats.

“Elevation of Privilege” cards beat all other cards. They can only be played when the player has no cards in the lead suit (or if the lead suit itself is “Elevation of Privilege”).

Aces are cards that allow finding unanticipated threats in the lead suit. The player must explain the threat itself.

When all cards have been played (all tricks completed), the player with the most points wins.

Annotate your diagram according to the threats identified.

You can pass a player’s hand to another between tricks. This allows specialized players to play cards that previous players did not understand.

Other players, besides the one announcing their card, can challenge it by finding the announced threat in other locations than those identified by the player. They gain an additional point.

Download “ Elevation of Privilege” and its extension “ Elevation of Privacy”, with the EoP rules.

Sources

• EoP, Creative Commons Attribution 3 License
• EoP (Privacy Extension), Creative Commons License
• EoPrivacy, Creative Commons Attribution 4 License

The Product Owner

The Product Owner is the representative of the clients and their needs. They are responsible for the proper progress of product development and the outcome of the project.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

They do not participate in development and are not part of the Agile team because they must remain neutral with respect to the team spirit that forms.

They are responsible for prioritizing tasks during ceremonies in order to maintain objectives. They are the main maintainer of the Backlog and the Lean Canvas.

They are often assisted by the Scrum Master.

They have the ultimate decision-making power and are the only one who can move a story to DONE (with the possible exception of specific bots for repetitive NFRs such as updates).

They are the interface between the team and other project stakeholders, and of course, the clients.

Product Owners can be organized hierarchically, with a Product Manager overseeing all the Agile teams of a project and the global Backlog. In this case, the Product Owner of a team manages only the stories of the Backlog with the labels assigned to that team.

The Scrum Master

The Scrum Master is responsible for the well-being of the team. They ensure that the team consistently follows the Agile “Scrum” method and its best practices.

At first, they lead the team’s daily “ Scrum” meetings.

They serve the team, making sure it feels good and remains effective. They work to remove obstacles the team encounters.

They ensure that the team becomes more autonomous. They are the one who introduces new practices when necessary, in agreement with the Product Owner.

They are by no means a kind of team leader.

Types of Agile Teams

An Agile team is made up of at most 7 people. Beyond that, the team’s efficiency quickly decreases.

The environment of these teams will be explained in more detail in a future article. But here’s a preview.

MarketOps

The operational marketing team has the ability to determine short-term marketing strategies and implement them quickly.

DevSecOps

Inherited from DevOps. This team develops, deploys (operations), and secures its developments.

It may sometimes be split into two teams, with one team dedicated to testing in order to reduce human biases caused by teamwork.

Blue Team / Red Team

For projects with greater resources, two teams can be dedicated to project security (IT or otherwise).

The Red Team represents aggression and attack. Its role is to think like a potential attacker and find every flaw in the project to compromise it.

The Blue Team represents defense. Its role is to defend the project against attacks and set up the necessary tools to achieve this.

This approach makes it possible to prevent rather than cure.

The Sprints of these teams can be organized around objectives:

• The CTF (Capture the Flag) consists of retrieving information within the system for the Red Team and preventing it for the Blue Team;
• The destruction arena consists of destroying portions of the system for the Red Team and preventing it for the Blue Team;
• “You shall not pass!” consists of closing system access for the Red Team and keeping them open for the Blue Team (useful for IT DDOS attacks);

Sources

• Scaled Agile Framework

The Lean Canvas allows you to quickly describe a project without overlooking important elements.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

It is used by startups to launch their projects but can also be applied in agile methodology to maintain project coherence among multiple stakeholders.

Indeed, a good Lean Canvas allows the agile team or teams to prioritize or rewrite stories, in agreement with the Product Owner, so as not to deviate from short- or long-term objectives desired by clients.

Overview of a Lean Canvas

• The “Problem” box allows defining the project’s objective. It helps keep the direction when designing User Stories.
• The “Solution” box allows defining the minimum viable objective to achieve before production. Development User Stories should be prioritized accordingly.
• The “Key Metrics” box allows defining the priority NFRs.
• The “Unique Value Proposition” box allows defining development User Stories and priority NFRs from the start of production.
• The “Unfair Advantage” box allows defining priority Spikes from the start of production.
• The “Channels” box allows defining marketing User Stories to prioritize.
• The “Customer Segment” box allows defining user personas (including buyers personas if the project involves marketing).
• The “Cost Structure” and “Revenue Streams” boxes can be the subject of NFRs. Their analysis helps prioritize certain stories (the stories that cost the most in the sprints where revenue streams perform best, and vice versa).

From a story perspective, the Lean Canvas is the visual representation of the Saga.

Download our Lean Canvas template (odg format, LibreOffice and OpenOffice).

Sources

• The Inventor of Lean Canvas

The unit of time in the agile Scrum method is the Sprint. It is a repeating period that can vary depending on the team, from 2 weeks to 1 month.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

It is during this period that the stories are completed.

It is framed by a kick-off ceremony that prepares the work and a closing ceremony that reviews the work done.

Each workday morning during this period, a daily meeting (or Scrum) takes place to keep the Sprint goals on track.

The goals of a Sprint are simple: move all the stories planned for it from TODO to DONE.

Special Case: Sprint 0

Sprint 0 is the very first Sprint, coming immediately after the Discovery & Framing phase. This Sprint mainly serves as a warm-up phase for the team.

The team gets familiar with the work environment, the flow of daily meetings, all project stakeholders, etc.

Contrary to what some sources suggest, I do not recommend describing the product and its target in this Sprint; the Discovery & Framing phase is more suitable for that.

This Sprint should resemble a normal Sprint as much as possible, while keeping in mind that the team is still discovering the subject.

Since this Sprint is not effective for actual project development, it is counted separately.

The Scrum (Daily Stand-up)

The Scrum (French: “mêlée”) is the daily meeting held every morning. It gives its name to this agile method.

This meeting should be held standing (also called a Stand-up meeting). It concerns only the development team and is facilitated (but not led) by the Scrum Master.

Each team member should have the opportunity to speak. They share what they did yesterday and what they plan to do today. If obstacles arise, the Scrum Master can decide either to resolve them immediately (if quick to fix) or schedule a discussion for another time.

The meeting should not exceed 15 minutes. It should occur at the same time and place every day (usually with a Scrum Board visible).

This meeting is also a social gathering for the team members. Gradually, the Scrum Master should step back to let the team take ownership of the meeting, reinforcing team spirit.

The idea is for team members to have a moment to help each other and maintain a global view of the project. It is not a reporting session (that can be automated and addressed one-on-one with the Scrum Master if needed).

The three questions this meeting answers are:

• What was done yesterday?
• What do you plan to do today?
• What obstacles did you encounter?

The Scrum of Scrums

If a project requires multiple agile teams, a scrum of representatives from each team (not the * Scrum Master*s) can be held. This scrum of representatives takes place after the team-level scrum.

Sometimes it is difficult to know who will actually use the project’s outcome. This is where the user persona comes in. They are the concrete representation of the project’s target users.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The Persona Sheet

A user persona resembles a role-playing game character sheet (for those familiar with RPGs). This sheet provides an at-a-glance view of all the characteristics of a person.

Through this sheet, we aim to understand their interests, history, culture… We create a fully imagined physical person.

Workshops in Discovery & Framing dedicated to the user persona bring this person to life, giving them substance. Feel free to use role-playing for this.

However, it is crucial not to include the person’s potential relationship with the project outcome, as this could distort their characteristics.

In Agile Marketing

There is no need to introduce this concept in marketing because it is well known. This sheet helps define acquisition channels and ways to reach the target.

It is to this persona that the Stories (or User Stories) are addressed. It is this target that performs the actions described within these Stories.

However, do not hesitate to modify or refine it during Sprints.

It may even need to be completely replaced if the project target changes.

In Agile Development

This sheet is useful for guiding usability and prioritizing the features requested by the client.

In alignment with the Product Owner, developers have a reference for discussion to best describe the User Stories.

Here too, this persona performs the actions described in the User Stories.

It is therefore useful to have multiple user personas depending on the project user roles. One user persona may represent an administrator, another a regular user.

In pure development projects (where the client handles marketing), user personas are defined in Discovery & Framing and validated by the client.

In projects including marketing, user personas can be modified by the marketing teams with validation from the development teams. In this case, the Buyer persona must also be included (the person who purchases the project outcome but does not necessarily use it).

It is even possible for a marketing team representative to become the Product Owner (or a deputy Product Owner) for the development team, since they are assumed to have the best understanding of the user personas.

Download our User Persona Template (odg format, LibreOffice and OpenOffice).

Sources

• Ricoh
• Just in Mind

The one-time pad is an encryption method invented by Gilbert Vernam, an American engineer at AT&T Bell Labs, in 1917.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

It is based on a much older encryption technique called the “ Vigenère cipher”.

The one-time pad was further perfected by the American general Joseph Mauborgne, who introduced the randomness of the encryption key.

It consists of encrypting a message with a key by adding the key to the message, element by element. The key must be used only once (hence the “one-time” or disposable). To decrypt, you subtract the key from the message.

Okay, history is nice but in practice, how does it work?

How it works

To understand the one-time pad, you first need to master (or at least understand) congruence (or modulo arithmetic).

The basics

We’ll start by using non-random texts for the keys. Note, this is just to start slowly because here security is zero.

We’ll take a Bigflo&Oli (french rappers) text as the message to encrypt and a Georges Brassens (french singer) text as the key.

The message is from “Alors alors”:

“Alors alors, On devait faire le tour de la Terre”

The key is from “Bancs publics”:

“Les gens qui voient de travers pensent que les bancs verts”

Got modulo in mind? Good, let’s start by assigning a number to each letter of our alphabet:

A=0, B=1, C=2, D=3, E=4, F=5, G=6, H=7, I=8, J=9, K=10, L=11, M=12, N=13, O=14, P=15, Q=16, R=17, S=18, T=19, U=20, V=21, W=22, X=23, Y=24, Z=25

So, for our 26-letter alphabet, we have numbers from 0 to 25.

Encryption

To encrypt, it’s simple, we add letters face to face. “Add letters, are you crazy?” you might say, but that’s why we’ve given them numerical equivalents, follow along. Since we haven’t assigned values to commas, we’ll remove them.

Let’s go:

"Alors alors On devait faire le tour de la Terre"

+

"Les gens qui voient de travers pensent que les bancs verts"

The “A” of “Alors” + the “L” of “Les” = A + L = 0 + 11 = 11 = L

Then “l” + “e” = l + e = 11 + 4 = 15 = p

So, little by little:

"Alors alors On devait faire le tour de la Terre"

+ + +

"Les gens qui voient de travers pensent que les bancs verts"

= = =

0 11 14

+ + +

11 4 18

= = =

11 15 32

= = =

L p ?

Indeed, for adding “o” and “s”, we get 14 + 18 = 32 and here is the problem, we don’t have a letter with value 32.

But remember, congruence (modulo, all that).

Since we have only 26 letters, we’ll use modulo 26 to stay in range.

Modulo 26 of 32 is:

$$ 32 \pmod{26} \equiv 32 -26 \pmod{26} \equiv 6 \pmod{26} $$

And 6 corresponds to letter G.

So, we can finish our calculation:

"Alors alors On devait faire le tour de la Terre"

+ + + ...

"Les gens qui voient de travers pensent que les bancs verts"

= = =

0 11 14

+ + +

11 4 18

= = =

11 15 32 mod(26)≡ 6 mod(26)

= = =

L p g …

In the end:

"Alors alors On devait faire le tour de la Terre"

+

"Les gens qui voient de travers pensent que les bancs verts"

=

"Lpgxw ndela Jb liitlx yrimi cw ishj hr eq Nicvw"

We have our encrypted message.

Decryption

To decrypt, we do the opposite: subtract the key from the encrypted message.

We have encrypted message “Lpgxw ndela Jb liitlx yrimi cw ishj hr eq Nicvw” and the key “Les gens qui voient de travers pensent que les bancs verts”.

So:

"Lpgxw ndela Jb liitlx yrimi cw ishj hr eq Nicvw"

- - -

"Les gens qui voient de travers pensent que les bancs verts"

= = =

11 15 6

- - -

11 4 18

= = =

0 11 -12

= = =

A l ?

Again, same problem as encryption, we have -12 with no corresponding letter.

We apply modulo 26:

$$ -12 \pmod{26} \equiv -12 + 26 \pmod{26} \equiv 14 \pmod{26} \equiv o $$

We continue the calculation and recover:

"Lpgxw ndela Jb liitlx yrimi cw ishj hr eq Nicvw"

- - - ...

"Les gens qui voient de travers pensent que les bancs verts"

=

"Alors alors On devait faire le tour de la Terre"

Mauborgne in da house

Okay, that’s nice but not very secure. To improve security, let’s listen to General Mauborgne and use a random key (or at least a more random one).

After putting a pseudo-random generator (yes, I know, an incomprehensible term, we’ll come back to it in a future article) on my computer, rolling my head on the keyboard, and letting the cat walk on it for entropy (yes, another incomprehensible term), I got this key:

"snchfzrhqjlnveivpyonrkudrpqtpxptfwrildof"

We’ll encrypt the following message, again by Bigflo&Oli, from “Dommage”:

“Il croisait cette même fille, avec son doux parfum”

Encryption

Here we go again:

"Il croisait cette même fille, avec son doux parfum"

+ +

"snchfzrhqjlnveivpyonrkudrpqtpxptfwrildof"

= =

8 11

+ +

18 13

≡ =

0 mod(26) 24

=

a y …

We get the encrypted message:

"Il croisait cette même fille, avec son doux parfum"

+

"snchfzrhqjlnveivpyonrkudrpqtpxptfwrildof"

=

"ayeythjhycnroxmhtksszvfhrkuvhlcwtqoxlutze"

Decryption

I think by now you get it. One quick example.

Still with modulo(26), decrypt this message:

"irszlg k wiu ladw"

with the key:

"xjgzpyisqbjmplcxhwfpijqrstghighnhylshqza"

And tweet or post the result.

The Che level

So far we’ve been cute, now we tackle the hard stuff.

Che Guevara and Fidel Castro communicated with a cryptographic technique based on the one-time pad. However, there was an extra complexity.

They didn’t use a sequence of numbers like we did, but assigned arbitrary numbers to letters of the alphabet.

Here is their substitution:

A=6, B=38, C=32, D=4, E=8, F=30, G=36, H=34, I=39, J=31, K=78, L=72, M=70, N=76, O=9, P=79, Q=71, R=58, S=2, T=0, U=52, V=50, W=56, X=54, Y=1, Z=59

You might ask “how do you do congruence with this?”, hold on, it’s coming.

Actually, they transmitted numbers, not letters, and the key was itself composed of numbers. They used the most known modulo (the one we unknowingly use), modulo 10.

So, how did they do it? Let’s take an example.

Encryption

We want to encrypt the message:

“J’aime pas les framboises” (“I don’t like raspberries” in french, just an example, I do like raspberries)

We replace the letters with their numbers:

“J A I M E P A S L E S F R A M B O I S E S” becomes “31 6 39 70 8 79 6 2 72 8 2 30 58 6 70 38 9 39 2 8 2”

We encrypt this message with the key:

"4317892093088287991744997181628757157172"

We add the numbers face to face and apply modulo 10. To ease things, they first grouped digits in packets of 5.

So, the message:

"31 6 39 70 8 79 6 2 72 8 2 30 58 6 70 38 9 39 2 8 2"

becomes:

"31639 70879 62728 23058 67038 93928 2"

We add with modulo 10:

"31639 70879 62728 23058 67038 93928 2"

+

"43178 92093 08828 79917 44997 18162 87571 57172"

=

747 (7+3 ≡ 10 mod(10) ≡ 0) …

=

74707 62862 60546 92965 01925 01080 0

The encrypted message is therefore:

"74707 62862 60546 92965 01925 01080 0"

Decryption

We subtract the key from the message using modulo 10:

"74707 62862 60546 92965 01925 01080 0"

-

"43178 92093 08828 79917 44997 18162 87571 57172"

=

"31639 70879 62728 23058 67038 93928 2"

Then, we convert the numbers into their respective letters. And now you’re thinking, “But how can they tell the difference between the letter with value 2 and the letter with value 23? How can they know if it’s a one-digit or two-digit number?”

Very simple, the digits used for one-digit numbers are never used to start a two-digit number.

In fact, if you encounter 3, 7, or 5, you know it’s a two-digit number and so you must group the following digit with it.

In our case, we start with a 3, so we know it’s a two-digit number, we join the 1 to the 3.

"31 639 70879 62728 23058 67038  93928 2"

Next, the 6 is not in (3,7,5), so it’s a one-digit number:

"31 6 39 70879 62728 23058 67038  93928 2"

And so on:

"31 6 39 70 8 79 6 2 72 8 2 30 58 6 70 38 9 39 2 8 2"

Finally, we replace the numbers with their corresponding letters:

"J A I M E P A S L E S F R A M B O I S E S"

There you go!

The Perfection of the Cipher

Claude Shannon (more name-dropping, but this one is the flagship of cryptographers) mathematically proved that the one-time pad is considered cryptographically secure, or unconditionally secure, during World War II. That is to say, the system cannot be cracked even if the adversary has unlimited resources or unlimited time, as long as they do not know the secret key.

Claude Shannon proved that the one-time pad is “perfectly secret,” meaning that if you have a ciphertext encrypted with the one-time pad, there is absolutely no way to guess anything about the original plaintext.

This means it is also resistant to brute-force attacks. If you try every key in the universe, you will get every possible message in the universe, so you have no way to know which one is correct.

I won’t redo the mathematical proofs here. This article is meant for broad, easy-to-understand dissemination.

But to achieve this perfection, there are some challenges.

The Difficulties

The difficulties to overcome to achieve a perfect one-time pad are as follows:

Key Size and Quantity

For every exchange, you need a new key. For every exchange, that is, each person you communicate with and each message you send to them.

So, if you exchange 2 messages with 2 people, that makes 4 keys. 3 messages with 3 people, 9 keys…

Moreover, the key must be longer than the message.

Because if it is shorter and repeated to complete the encryption, you increase the probability of certain letters appearing and you can see patterns emerge giving clues about the message. This problem is similar to the need for randomness in creating keys.

This poses problems for securely storing keys, as the cipher collapses if the key is recovered by the adversary.

The Randomness of the Keys

In French, the letter “e” is the most commonly used letter, then “a”, then “s”…

And patterns like “le”, “les” are frequent. The key, when repeated, increases the chances of encrypting the same patterns in the same way.

The attacker can spot these patterns and deduce when the key is repeated.

Gradually, they can reconstruct the key or even manage without it.

An example:

Imagine the key “Wesh”, the attacker doesn’t know it but has found its length is 4 by analyzing the frequencies of recurring patterns.

They can cut the ciphertext into groups of 4 letters: “Uedl lsll ysfk wqfl myaz kvlp newu zsmj a” and subtract the groups from each other:

Uedl - lsll = X? + “Wesh” - (Y? + “Wesh”) = X? - Y? + “Wesh” - “Wesh”

= X? - Y?

The key disappears and we have two texts which are no longer random subtracted from each other. We analyze the letter frequency and form hypotheses. The letters resulting from the subtraction, the most frequent ones, are more likely to correspond to an “e” in one of their base letters, etc.

Step by step, by multiplying hypotheses, possibilities reduce and one can find the key or decrypt messages without it.

In conclusion, if you do a frequency analysis running your computer for three days, you will get the following message:

“Y’a le pote condamné qui sortira en douce,” always Bigflo & Oli, excerpt from “Comme d’hab”

This seems obscure, incomprehensible. Don’t worry, a future article will clarify all this with examples.

So, to ensure every letter has the same probability of appearing, you need randomly generated keys. But that is very hard to do. We can make pseudo-randomness but as its name indicates it’s not truly random, the probabilities of letter appearances are not exactly equal.

Key Transmission

Key transmission is not a problem for very small keys (an article will soon explain how).

But it becomes very complicated once keys must exceed a certain size (the current minimum size is 4096 bits or, in some crazy cases, 8192 bits, that is 1024 bytes, 1 KB for IT folks).

So, often, keys are exchanged physically beforehand. This is not practical and becomes very complicated when sending many messages to many interlocutors.

Uniqueness of Use

We return to the same problem as with key size and randomness.

If you reuse a key, you lose the randomness and the ciphertext becomes again vulnerable to frequency analysis.

The problem is even more serious because uniqueness of use implies destroying the key after use and there is no easy way to ensure that your correspondents do not keep the keys in a corner at risk of being stolen later (in 5 years, 10 years…) or reuse them for other messages on their side.

Message Poisoning

The last possible problem is key poisoning but this can be easily fixed (at least temporarily) in IT.

Since the one-time pad is immune to brute force (all keys in the universe produce all messages in the universe), an attacker can mess with the ciphertext to give it a different meaning.

They can also modify the key without necessarily knowing it and thus alter the plaintext.

Example:

Let’s take our famous message again:

“J’aime pas les framboises”

which we encrypt with the key:

“nzhtjplgpggsefirksxohsbyhkazsb”

using modulo 26 (thus without the apostrophe), we get:

“N zrry rrg yef eytvqzohky”

The attacker only knows this secret message but can modify the encryption key. They replace the key by:

“ezjfuqjcltbmtcvihgdtgefirksxohsbyhkazsb”

And that gives the following plaintext:

“Jaimebienlesfraisiers” ("I like strawberry bushes" in french)

So, not at all the original message.

To prevent this, in IT, you can sign and hash the message and keys so a user can verify that the message they receive was indeed created by the correct key and that the key they use is still the correct one.

We will come back to how signing and hashing work in future articles.

I think we have already well exceeded 2000 words. Let’s stop here.

Contrary to what its name suggests, the Vigenère cipher seems to have been invented by Giovan Battista Bellaso, an Italian cryptographer from the 16th century.

Nevertheless, it is the Vigenère cipher that endured, not Bellaso’s.

Blaise de Vigenère introduced his cipher 20 years after Bellaso, with some improvements.

This encryption method began to be called the Vigenère cipher from the 19th century onwards, without regard to Bellaso. Moreover, Bellaso was quickly forgotten in favor of Porta.

Lots of names. Ah, history!

Of course, this cipher is not secure and was broken as early as the 19th century.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The Key Instead of the Shift

The Vigenère cipher introduced a major innovation compared to the Caesar cipher: the concept of a key.

This difference classifies this cipher as polyalphabetic rather than monoalphabetic. It is no longer “one plaintext letter gives one and only one ciphertext letter,” but a plaintext letter can produce two different ciphertext letters, and a ciphertext letter can come from two different plaintext letters.

Therefore, multiple substitution alphabets are used (different letter shifts within the same message), unlike the Caesar cipher. Hence the term “polyalphabetic.”

The substitution rules (how you switch from one alphabet to another) are given by the letters of the key and a substitution table.

Don’t quite get it? We will illustrate this with examples in the following chapters, no worries.

Also, another article will cover the classification of ciphers.

The Cipher Itself

Let’s get to the heart of the matter. There are two ways to represent the classic Vigenère cipher (with variants being another story).

Table Representation

For the table representation, we need a table (yes, really), or rather a substitution matrix. Here it is:

It may look dense but it’s not so complicated. If you look at the rows, you see the Latin alphabet shifted by one letter per row. Similarly for the columns.

The cipher works by taking the letter at the intersection of the plaintext letter (columns) and the key letter (rows). That intersection letter is the ciphertext letter.

Example:

Encryption

Take the message: “J’ai plaqué mon chêne” (excerpt from “Auprès de mon arbre” by Georges Brassens)

And the key: “Una mattina” (excerpt from “Bella Ciao”)

Look at the intersection of each letter:

Column “J” and row “U” give “D”, column “a” (ignoring the apostrophe for now) and row “n” give “n”, column “i” and row “a” give “i”, etc.

If the message is longer than the key, repeat the key. Consider “ê” and “é” as “e”.

We get: “D ni bltjcr mia ctegx”

Decryption

For decryption, for each row corresponding to the key letter, find where the ciphertext letter is located and take the letter at the top of that column, which is the plaintext letter.

So, taking the key: “Una mattina” And the ciphertext: “D ni bltjcr mia ctegx”

In row “U”, letter “D” is at the 10th position, so column “J”, in row “n”, letter “n” is at the 1st position, column “a”, etc.

We recover: “J ai plaque mon chene”

Congruence Representation

The second representation is mathematical: congruence. If you don’t know this word, check this article.

To use congruence, substitute letters with numbers using the rule:

A=0, B=1, C=2, D=3, E=4, F=5, G=6, H=7, I=8, J=9, K=10, L=11, M=12, N=13, O=14, P=15, Q=16, R=17, S=18, T=19, U=20, V=21, W=22, X=23, Y=24, Z=25

To encrypt, add the key to the message. To decrypt, subtract the key from the message.

Example:

Encryption

Take the message: “J’ai plaqué mon chêne” And the key: “Una mattina”

Align the message and key (removing apostrophes and accents) and repeat the key to cover the message:

"J ai plaque mon chene"
+
"Una mattina Una matti"
=
9
+
20
=
29

There’s no letter with value 29 (max is 25), so we use congruence modulo 26 (numbers 0 to 25).

Thus,

$$ 29 \equiv 29-26 \pmod{26} \equiv 3 \pmod{26} = D $$

We continue calculating:

"J ai plaque mon chene"
+
"Una mattina Una matti"
=
9
+
20
=
29
≡
3 mod(26)
=
D

And so on. We get the ciphertext: “D ni bltjcr mia ctegx”

Decryption

To decrypt, subtract the key from the ciphertext, again using congruence.

"D ni bltjcr mia ctegx"
-
"Una mattina Una matti"
=
3
-
20
=
-17
≡
-17+26 mod(26)
≡
9 mod(26)
=
J

Step by step, we return to: “J ai plaque mon chene”

Variants

The Vigenère cipher has known many variations. Here are a few:

The Bellaso Precursor

Before Vigenère, there was Bellaso. But the poor man was overshadowed by another Italian, better known in his time, Giovanni Della Porta, and later by Vigenère. So sometimes it’s called the Porta/Bellaso cipher.

This cipher uses the following table:

The left column contains the key letter, and the right column contains the substitution.

A quick example to show how it works:

Encryption

Take the message: “Trois Anneaux pour les Rois Elfes” (excerpt from Tolkien’s “The Lord of the Rings” in french)

With the key: “La République Galactique” (excerpt from Lucas’s “Star Wars” in french)

The first key letter is “L”.

Look for this letter in the first column; “L” is in the sixth row with “K”.

The first letter of the message is “T”. Find “T” in the second column on the sixth row.

You see that “T” is below “L” (in the two lines present in each second column row), so the ciphertext letter is “L”.

Next, the second key letter is “a” (to make sure you’re following).

It is in the first row of the first column. The second plaintext letter is “r”.

In the second column of the first row, “r” is below “e”, so the ciphertext letter is “e”.

Then:

• “R”: 9th row: “o” becomes “j”,
• “e”: 3rd row: “i” becomes “t” (“i” is above “t”),
• etc.

The ciphertext is: “Lejtm qafnsem fbme xvb ijyh zyxpm”

Decryption

Use the key to find the row and match. So:

With the key: “La République Galactique” And ciphertext: “Lejtm qafnsem fbme xvb ijyh zyxpm”

• “L”: 6th row: “L” becomes “T”,
• “a”: 1st row: “e” becomes “r”,
• etc.

You recover the message: “Trois Anneaux pour les Rois Elfes”

Beaufort

We’ll quickly cover this and the next variant. The only difference from Vigenère is that instead of adding the key to the message to encrypt, you subtract the message from the key.

So to decrypt, subtract the ciphertext from the key:

Key – Message = Ciphertext ⇒ Message = Key – Ciphertext

Of course, this is in the congruence representation.

German Variant of Beaufort

For this variant, it’s the opposite. Subtract the key from the message to encrypt.

To decrypt, add the ciphertext to the key:

Message – Key = Ciphertext ⇒ Message = Key + Ciphertext

Disordered Alphabet

You can shuffle the letters of the alphabet. If both correspondents have the same table, the system works.

Of course, each letter must appear only once per column and row.

Also, all letters must appear in every column and row.

For example, consider these values:

C=0, E=1, A=2, Z=3, B=4, D=5, F=6, G=7, H=8, I=9, J=10, K=11, L=12, M=13, N=14, O=15, P=16, Q=17, R=18, S=19, U=20, T=21, V=22, W=23, X=24, Y=25

Expanded or Different Alphabet

We worked with a 26-letter Latin alphabet but nothing stops you from expanding it to include extra symbols or using a different alphabet.

For example:

A=0, B=1, C=2, D=3, E=4, F=5, G=6, H=7, I=8, J=9, K=10, L=11, M=12, N=13, O=14, P=15, Q=16, R=17, S=18, T=19, U=20, V=21, W=22, X=23, Y=24, Z=25,!=26,?=27

In this case, we have modulo 28 instead of 26.

Bellaso Variants

We used Bellaso’s original table but nothing prevents modifying it.

You could have three letters in the first column instead of two and reduce the number of rows accordingly.

Or have a different number of letters per row.

Or change the alphabet substitutions in the second column.

The only requirement is that your alphabet’s number of letters/symbols is even so each symbol has a match in the second column.

Weaknesses

Frequency Analysis Attack

As mentioned in the historical preamble, this cipher and its variants were broken in the 19th century.

It suffers from the same weakness, though much more resilient, as the Caesar cipher.

Languages do not use letters with the same frequencies. For example, in French, the letter “e” is much more frequent than others.

An attacker can exploit these frequency differences to guess the message or the key.

But unlike Caesar’s cipher, there is a preliminary step.

The attacker must first try to guess the length of the key. This allows segmenting the ciphertext into equal-length segments.

To do this, they look for patterns in the ciphertext — groups of three letters or more that appear regularly.

Languages also have common patterns: for example, “les” appears frequently in French.

It’s more likely the same part of the key falls on the same plaintext pattern, encrypting it the same way, than by pure chance.

By spotting these patterns, the attacker estimates the key length and can segment the ciphertext.

Each plaintext letter in a segment is encrypted by the same key letter, allowing frequency analysis.

This might seem obscure or confusing. Don’t worry, a future article will clarify this with examples.

The Vigenère cipher is halfway between the Caesar cipher, from which it takes the substitution rule, and the one-time pad, which is the perfect algorithm, appearing only at the beginning of the 20th century.

The Caesar cipher, despite its name, was apparently already used by the Spartans, so well before Julius Caesar.

It’s a variation of shift cipher, which is a substitution cipher. Complicated words for simple concepts, you’ll see.

Augustus used another variant of the Caesar cipher.

It’s very possible that it inspired the Vigenère cipher.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The big picture: substitution

Let’s start with the broadest concept. Substitution ciphering consists of replacing one letter with another. Plain and simple.

This technique is the most widespread in history and it is different from permutation, which consists of changing the order of the letters in a text.

Modern algorithms use both methods at once because substitution alone is not very secure cryptographically (except in the very specific case of a one-time pad).

The medium picture: the shift

Among substitution cipher methods, the simplest is the shift method.

It’s a monoalphabetic ciphering method (another complicated term for a simple concept). That means that one letter is associated (during encryption/decryption) with exactly one other letter. Two different letters cannot result in the same letter after encryption/decryption, and likewise, one letter cannot produce two different letters after encryption/decryption.

For mathematicians, this is a bijective relation.

The shift method therefore consists in shifting a letter within the alphabet.

The small picture: the Caesar cipher

Nowadays, the term “Caesar cipher” refers to all these shift cipher methods, but more precisely to a shift by 3.

Let’s take the Latin alphabet, the one used by Caesar (even though it seems he preferred the Greek alphabet for his secret messages), and assign values to the letters:

A=0, B=1, C=2, D=3, E=4, F=5, G=6, H=7, I=8, J=9, K=10, L=11, M=12, N=13, O=14, P=15, Q=16, R=17, S=18, T=19, U=20, V=21, W=22, X=23, Y=24, Z=25

We apply a shift of 3:

A=0+3=3=D

B=1+3=4=E

…

However, you can see that if we do:

X=23+3=26

we don’t know what letter 26 is, unless we use congruence. If this word is unfamiliar, read this article.

We’ll consider the Latin alphabet in modulo 26. So, if we redo the calculation:

X=23+3=26 ≡ 26-26 *mod*(26) ≡ 0 *mod*(26) ≡ A

So, in the Caesar cipher, A becomes D, B becomes E, X becomes A, Y becomes B, Z becomes C…

We’ve shifted the letters by 3 positions.

An example:

Consider the message: “CAVE CANEM” (“Beware of the dog”, in Latin)

We apply the Caesar cipher:

“C A V E C A N E M”

=

2 0 21 4 2 0 13 4 12

+3

=

5 3 24 7 5 3 16 7 15

=

“F D Y H F D Q H P”

Here’s your encrypted message. And to decrypt it, we do the opposite: subtract 3 from the letters of the encrypted message.

It’s simple, isn’t it? And you can see we’re at cryptography level 0.

But at the time, it seemed good enough.

Variants

Of course, variants of this cipher have been developed throughout history.

ROT13

This variant consists of shifting by 13 letters instead of 3. “ROT” is short for rotation, because a shift with a modulo can be seen as a rotation.

This shift is special because it’s symmetric for a 26-letter alphabet (13 being half of 26). That means that if you encrypt a message twice with this shift, you get the original message back.

Example:

First encryption:

“C A V E C A N E M”

=

2 0 21 4 2 0 13 4 12

+13

=

15 13 34-26 17 15 13 26-26 17 25

=

15 13 8 17 15 13 0 17 25

=

“P N I R P N A R Z”

Second encryption:

“P N I R P N A R Z”

=

15 13 8 17 15 13 0 17 25

+13

=

28-26 26-26 21 30-26 28-26 26-26 13 30-26 38-26

=

2 0 21 4 2 0 13 4 12

=

“C A V E C A N E M”

Tadaa, we recover “Cave Canem”.

Augustus

I said earlier that the Caesar cipher was useless. Wait until you see this one. Forget about modulo, that’s too complicated, and counting to 3 is also too complicated—just count to 1, that’s fine.

Augustus (Caesar’s nephew) used a shift cipher of 1. Yes, you read that right—1. Oh, and what about “z”? Remember, no congruence. So “z” becomes “aa”—yes, two “a”s.

Do I really need to give you an example?

7up, K9

Of course, there are also ciphers based on wordplay, invented for children’s games.

“7up” is a shift of 7.

“K9” is a reference to the robot dog from Dr Who (and also a very good mail app for Android…). It’s a pun on the word “canine” of course. It’s a shift of 1 backward (25 forward).

Extending the alphabet

You can, of course, use an alphabet other than the Latin alphabet, or even take an arbitrary order of symbols. All that’s required is that the symbols be ordered.

Example:

Let’s take this fictional alphabet:

“: ; ! , t 2 5”

If we apply an increment of 3, we get:

“, t 2 5 : ; !”

How bad this is

By now you understand just how bad this system is. Let’s look at two methods to break a cipher made with this technique.

Frequency analysis

Every language has letters that appear more frequently than others. For example, in French “e” is (by far) the most frequent letter. So, an attacker just needs to analyze the frequency of symbols in the message. If the encrypted message contains a lot of “g”s and the parties are French, the attacker can hypothesize a shift of 2 and easily recover the original text.

Brute-force attack

For a 26-letter alphabet, there are 26 possible combinations. It’s really not hard to test them all.

Testing all combinations is called a brute-force attack. It’s the dumbest (the most brute) approach: you test every possibility. But it’s also the attack that requires the most resources, since you’re testing everything.

Congruence was formally studied for the first time at the end of the 18th century and the very beginning of the 19th century by Carl Friedrich Gauss, the “master” of mathematicians.

He gave his name to the bell curve known as the “Gaussian curve.”

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Before that, people used congruence without being aware of its properties, just as we know how to add 2 carrots and 3 carrots without realizing it’s a sum in arithmetic.

That’s all very nice, but… what is it, exactly?

The principle

Congruence is a mathematical structure. That is, a set of rules, that allows us to relate integers to one another within a finite set.

That is, if we consider a congruence of modulus (or modulo) n (we’ll see later what a modulus is), we have the following properties:

$$ a \equiv a \pmod{n} $$ $$ \text{if } a \equiv b \pmod{n} \quad \text{then } b \equiv a \pmod{n} $$ $$ \text{if } a \equiv b \pmod{n} \quad \text{and } b \equiv c \pmod{n} \quad \text{then } a \equiv c \pmod{n} $$ $$ \text{if } a \equiv b \pmod{n} \quad \text{and } c \equiv d \pmod{n} \quad \text{then } a+c \equiv b+d \pmod{n} \quad \text{and } a \times c \equiv b \times c \pmod{n} $$

And here you’re probably thinking: “I don’t get it. Why are there three bars in that equal sign? What does mod mean? So many questions!”

Don’t worry. We’ll go through examples and clearer illustrations.

Looping around

Another way to think about congruence and modulo is as a loop or cycle.

A small example: minutes

Imagine you’re looking at a clock and paying attention only to the minute hand.

You want to add 45 minutes and 30 minutes, but you don’t care about the hour (just the minute hand). What do you get?

15 minutes, right?

Congratulations—you’ve just done a modulo 60 congruence calculation.

And you can add numbers much larger than 60 (or any other modulus). You simply subtract 60 from the result as many times as you can. The result must lie within the set (here, between 0 and 59).

How much is 59 + 258 minutes modulo 60?

Answer at the end of this blog post.

A second small example: hours

Now, with hours. You look only at the hour hand (ignore the minutes). You add 5 hours to 8 hours. What do you get? Careful, it’s a clock.

1 o’clock, right? (since a clock only shows 12 hours)

You’ve just done a modulo 12 congruence calculation.

So, why an equal sign with three lines? Because in this second example, the result is not equal to 1 in the strict sense. We removed some extra hours. So we write it with this sign called “congruent to.”

You can see that we can add huge numbers while keeping the result within a limited set of numbers (a finite set). The size of the congruence limits the possible results. Here, for example, the set contains 12 numbers, the 12 hours of a clock.

Of course, since it’s a cycle, it also works with subtracting hours and with negative integers. You simply add the modulus afterward to bring the result back into the set (here, between 0 and 11).

So, how much is 2 – 18 hours modulo 12?

Answer at the end of this blog post.

Another example: fingers

If you consider the fingers of one hand, you have a congruence of 5 (usually), and with two hands, a congruence of 10.

Not convinced?

Take just one hand and count with your fingers: 3 + 4. How many fingers remain?

2, I think.

Now, take two hands and count 8 + 9. How many fingers remain?

7, right?

I’m deliberately using the term “remainder.” You’ll see why.

And the remainder?

You can also think of congruence as the remainder of a division (Euclidean division, for those who know the term).

The denominator of the division (the number below) is the modulus, and when you’ve finished dividing, what’s left is the remainder.

Note: it’s important to remember that this works with integers, so no decimals. You stop your division before the remainder becomes a decimal.

So, if we divide 3 by 2, what remainder do we get?

1, of course.

And if we divide 5 by 2, what remainder?

Also 1. Not bad.

So that means 1 ≡ 3 mod(2) and 1 ≡ 5 mod(2), right?

And therefore:

$$ 1 \equiv 3 \pmod{2} \equiv 5 \pmod{2} $$

Now, are the remainders of 33 ÷ 26 and 85 ÷ 26 congruent?

Answers

How much is 59 + 258 minutes modulo 60?

$$ 59+258 = 317 \equiv 317-60 \times 5 \pmod{60} $$

(we have 5 times 60 that fits in 317)

$$ \equiv 317-300 \pmod{60} \equiv 17 \pmod{60} $$

So:

$$ 59+258 \equiv 17 \pmod{60} $$

How much is 2 – 18 hours modulo 12?

$$ 2-18 = -16 \equiv -16+12 \pmod{12} \equiv -4 \pmod{12} $$

(it’s still negative, so not between 0 and 11)

$$ \equiv -4+12 \pmod{12} \equiv 8 \pmod{12} $$

So: $$ 2-18 \equiv 8 \pmod{12} $$

Are the remainders of 33 ÷ 26 and 85 ÷ 26 congruent?

$$ 33/26 = (26+7)/26 $$

so the remainder is 7.

$$ 85/26 = (26+59)/26 $$

59 is greater than 26, so we continue:

$$ = (26+26+33)/26 $$

well look, 33, just like in the first division—so we continue:

$$ = (26+26+26+7)/26 $$

so the remainder is 7.

Therefore, yes, the remainders of 33 ÷ 26 and 85 ÷ 26 are congruent.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Step 1, the basics

We start at the beginning. The very first step is the password, the most common authentication method.

What makes a good password, how to best manage the multitude of passwords.

All about a good password

For developers, this article explains how to manage password authentication on your system. Note, the recommendations in this article are valid for the first quarter of 2019. If you read this article in 2038 (well hello to transdimensionals already, or “−··· −−− −· ·−−− −−− ··− ·−· ·−−·− ·−·· ·−−−−· ·· −· − · −· − ·· −−− −· −·· · ·−·· ·− −−· ·−· −−− ··· ··· · ···− −−− ·· −··−  " in your language), the recommendations will probably (surely) be outdated.

Passphrase implementations

Step 2, strengthening

Once the password is set (so no more “1234567890”), we move to the second step (which is aptly named).

Two-factor authentication is based on the principle “Something you know, something you have.”

All about two-factor authentication

Step 3, distributing authentication

Once authenticated on one system, why do it again on others? It is error-prone and not very pleasant.

There is a solution that allows you to securely distribute your authentication. With this solution, you authenticate to one service and other services, systems, websites you use can recognize your identity and authenticate you on demand without repeating the first two steps.

Social logins, how do they work?

Step 4, access control

Once authenticated, you do not have access to all the resources provided by services either. There are rules to select what you can access and what is denied. Users of a service are organized according to groups or roles. What on social networks is called tribe, clan, horde, community, committee, section, commission, (·−−· −−− ·−·· −·−− ·−−· ···· −−− −· ·· · ··· for the transdimensionals)…

What is a community? Groups and roles

There you go. Of course, this is a base. There are many variants and special cases.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The service implements access control. Most commonly, it uses group-based or role-based access control. For a social network, there is also relationship-based access control.

Group-Based Access Control

This is the tribe.

This type of access control is a grouping of users who have access to a certain set of resources.

This grouping defines permissions (rights) for using these resources. Each user in this group inherits those permissions.

Role-Based Access Control

This is the tribal chief or shaman…

This type of access control defines a set of permissions (rights) that are collectively assigned to a specific user.

Typically, there’s an Administrator role that has all permissions. Other roles are granted to users by this Administrator. A user cannot assign a role to themselves.

Roles can also be hierarchical. A higher-level role includes all permissions of a lower-level role plus additional ones.

Difference and Relationship Between the Two

When comparing these two types of access control, it can be tricky to distinguish them. Depending on the system, the distinction may not even be explicitly made.

To clarify, here is the distinction:

• Groups are generally used to classify resources and users. They manage identity. For example: a user belongs to one or more groups.
• Roles are generally used to manage permissions and rights. They manage activity. For example: a user can perform a certain action because they have a specific role.

Groups and roles are linked because they complement each other: A group defines the scope of a role. That is, the users and resources on which the role can act. But a role can exist across multiple groups.

Example: Facebook

You might be a member of the “Raspberry Fans Group” and like the “Blackberry Fans Page.” From an access control perspective, you belong to two groups.

Within the “Raspberry Fans Group” you’re a member, and within the “Blackberry Fans Page” you’re a moderator. These are your roles.

As you can see, your roles are bounded by groups. You are a moderator of the “Blackberry Fans Page” only, not elsewhere. But there might be a moderator of the “Raspberry Fans Group” who has the same permissions there as you have on the page. Thus, a role can be present in multiple groups.

Also, group membership provides default permissions. As a regular member of the “Raspberry Fans Group”, you have access to that group’s posts. You wouldn’t have this access if you weren’t a member.

Example: A Computer

The distinction between group and role is subtle (especially on UNIX). But you can think of the computer itself as the group, and the user accounts on that computer have roles (usually one is the administrator—a role that exists on all computers).

Relationship-Based Access Control

This refers to your friends or contacts on social networks.

The existence of a relationship between you and a friend grants permissions to both of you over each other’s resources.

The relationship manages permissions. These can be symmetric (the friend has the same rights as you) or asymmetric (the friend has fewer permissions—this is closer to a “fan” or hierarchical relationship).

Capability-Based Access Control

This type of access control allows for fine-grained permission management.

Permissions are attached to a specific resource (object) rather than a group of resources (as in group- or role-based control). The object-permission pair forms a capability.

If a user possesses the corresponding capability, they can access that object with the defined permissions.

This allows for much more precise permission management since it’s handled resource by resource.

This control type is used by OAuth2 (the mechanism used for social login) to manage permissions. For example, when you use Facebook social login on a website, Facebook tells you that the site wants to access your email, your friends list, or other data.

This is the capability you delegate (fully or partially) to the site over the Facebook resource (which in this case is your profile).

The underlying protocol that everything relies on is OAuth2. So, let’s start there.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

OAuth2

If there’s a “2”, that means it’s the second version.

The first version, OAuth, had multiple security vulnerabilities. Never use it.

OAuth2 is the protocol that manages authorization. It allows a device to access resources—data stored on a server.

To do this, it uses scopes. These are parameters that request access to specific resources.

For example, the scope “profile” requests access to the user’s profile information.

The server returns a claim, which represents the requested data.

For the “profile” scope, the claim would be:

{
  "name": "Alice the rabbit",
  "firstname": "Alice",
  "lastname": "the rabbit"
}

This protocol can handle different methods of granting authorization, but for social login, only one concerns us.

Authorization Code Grant

This authentication method allows a service to authorize a user’s login without requiring their password, by trusting an authorization server.

Here’s how it works:

• The user wants to log in to a service
• The service redirects the user to the authorization server’s login page
• The user logs into the authorization server if not already logged in (they enter their password at this step on the authorization server)
• The authorization server displays a confirmation panel asking the user if they really want to connect to the service
• The user confirms
• The server then redirects the user back to the service with an authorization code
• The service uses the received authorization code to request an access token from the authorization server
• The authorization server approves the connection between the service and the user
• The user is logged in to the service

As you can see, the user never enters their password on the service. Password storage happens only on the authorization server.

OpenID Connect

This extension of OAuth2 is managed by the OpenID Foundation, which also maintains the original OpenID with its own mechanism.

This extension handles authentication (identity). It leverages the authorization server as an authentication server.

Building on the “Authorization Code Grant” flow, it verifies the user’s identity with the server using information returned by it (a unique identifier, usually an email, though ideally a UUID).

Additionally, this extension offers several options:

• The UserInfo endpoint provided by the authentication server: allows dynamic retrieval of user profile information
• The Session endpoint provided by the authentication server: enables session verification to synchronize sessions (e.g., making sure the user’s session on the service ends when they log out of the server)
• A logout flow that logs the user out of all services when they log out of the authentication server
• An implicit login flow called “Single Sign-On” which allows users to log into a service seamlessly. The service and the server appear unified as a single application.

Advantages

This system links multiple services without requiring a password for each one, enhancing security.

It also enables data synchronization between services with a central information silo (a reference point for data storage). This reduces data conflicts when properly implemented.

Risks

Using this system requires full trust in the authorization provider. The provider can access information within connected services because it can impersonate the user and obtain an authorization code on their behalf.

Identity and data synchronization isn’t always well executed. You might end up with multiple accounts on the same service just because you changed your email on the authorization server. The service relying on email won’t recognize the old account and will create a new one.

Logout is often not implemented. Major social login providers use persistent sessions, and many services don’t implement proper logout checks. As a result, services may remain logged in (the user stays connected) even after logging out of Facebook or Google.

If the authorization server is attacked, compromised, or weakened, all linked services become vulnerable.

Attacks

Clickjacking

A hacker could create a malicious service that loads the authorization server inside a transparent iframe.

To simplify, imagine your authentication server as a smartphone.

You tap the center button on your smartphone screen to accept the service.

With clickjacking, the hacker has placed a transparent sheet over your screen, and when you click the button, you’re actually clicking on the sheet, unknowingly granting the malicious service access to your data.

For developers: Implement the X-Frame-Options header with strict settings (e.g., “SameOrigin”) and/or use JavaScript framebusting techniques.

Cross-Site Request Forgery (CSRF)

A hacker can trick the user into clicking an authorization button that redirects, not to the intended service, but to the hacker’s malicious service. The malicious service captures the authorization code and then redirects the user to the correct service.

The user notices nothing, but the malicious service now has the authorization and access tokens.

For developers: Enforce fixed, pre-registered redirect URLs for services.

There are other CSRF variations; I’ll detail them in a future article.

Confused Deputy Problem

This issue occurs when services do not correctly verify the identity of the user utilizing the authorization server.

A malicious user can log in to the authorization server and trick the service into thinking they are another user by manipulating the information used to identify the user.

If the service doesn’t use a reliable identifier for proper user identification, it gets fooled and grants access to the wrong account.

Example:

A service uses an authorization server that provides both a UUID and a username.

The authorization server has two accounts:

• Alice with UUID: 46d07175-dbf2-46e2-80fc-c6493e481479
• Oscar with UUID: 362b862b-51b3-41f4-82c7-82eb677a9aa4

The authorization server provides UUIDs as unique identifiers, expecting the service to use them.

But instead, the service chooses to use the username as the unique identifier.

Oscar changes his username to “Alice” (allowed by the server since it uses UUID as the true unique ID).

Now, when Oscar logs into the service, he gains access to Alice’s account (because the service mistakenly uses username as the unique identifier).

For developers: Always choose a truly unique identifier. UUIDs are designed for this purpose. They don’t carry user information and allow users to freely change their details (even email) without affecting uniqueness.

The idea behind two-factor authentication is to ensure that the user identifies themselves using two techniques that are very unlikely to be compromised at the same time.

The key phrase to remember is:

“Something you know, something you have.”

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

In practice, this often involves a password (“something you know”) and a physical device (e.g., a message on a smartphone, “something you have”).

Two-factor authentication is just one method within a broader concept: multi-factor authentication.

Multi-Factor Authentication

When you watch a spy movie with a secret base, you sometimes see the hero or villain scan their iris, type a code on a keyboard, place their hand on a pad, and even spit into a sensor before opening the door. That’s multi-factor authentication.

You may encounter or implement an authentication chain, meaning several authentication steps are required, and if one fails, authentication stops.

In some real-world high-security scenarios, more than two steps are used. But most of the time, two steps are enough.

Methods (and Their Pitfalls)

SMS

The most common two-factor authentication method is SMS.

This is the weakest method because it’s easily hackable. With enough resources, a hacker can compromise SS7.

SS7 is the network layer that routes SMS messages. Designed in the 1970s, it’s long outdated and has been exploited in real attacks ( NextImpact article in french).

It’s also inconvenient in areas with no mobile network coverage.

PUSH

Another method used by some services is push authentication. This involves a notification sent to your smartphone asking for your approval. The message is pushed by the service to your device.

This requires an internet connection on your phone, which can be a weakness because hackers can attempt to intercept these notifications.

HOTP/TOTP

The second most common method is HOTP/TOTP (HMAC-based One-Time Password / Time-based One-Time Password).

• HOTP (“One-time password based on HMAC”) generates a unique password using a cryptographic HMAC and a counter.
• The user installs an app on their smartphone that generates a 6-character token on demand. This token is generated using:
  • A seed stored both in the web service account and the app.
  • A counter that increments in sync between the service and the app.

When the user enters the token, the web service compares it with its own calculation. If they match, authentication succeeds. The service knows it’s from the user’s smartphone because only that device holds the seed.

The seed is usually transferred from the service to the user via a QR code.

• TOTP (“Time-based One-time Password”) extends HOTP by using the current time (typically 30-second intervals) instead of a counter. TOTP is much more widely used than HOTP.

We recommend using FreeOTP to generate tokens on smartphones or other devices:

• Android and F-droid
• iOS
• Source Code

These methods do not require internet or SMS connectivity, making them safer. Hackers can’t intercept messages because no messages are sent.

Hardware Devices

The most secure method is to use a dedicated hardware device. During setup, the device is linked to your account on the web service. Thanks to cryptographic recognition (the device contains an asymmetric key), the service recognizes it as “something you have” for authentication.

The downside: you have to carry an extra device—and not lose it!

Some banks also provide hardware tokens tied to your bank card, often using TOTP internally.

Recovery Codes

While not strictly “something you have,” recovery codes are the best backup method if you lose your device or smartphone. It’s a list of single-use tokens you can use instead of your primary factor.

Store this list in a very secure location and only use it as a last resort.

Attacks

These solutions aren’t perfect and can be targeted by attackers.

Social Engineering

In SMS-based two-factor authentication, attackers can use social engineering. For example, DeRay McKesson’s case.

Hackers impersonate the phone owner with the carrier, request a SIM card swap, and once their SIM is activated, they receive all of the victim’s SMS messages.

Proxy Attacks

Attackers can also intercept via proxies, either by:

• Creating a fake mobile network nearby to intercept SMS ( NextImpact article in french), or
• Setting up a phishing site that mimics the real one to capture credentials and tokens. This is known as a homograph attack or typosquatting.

Mitigation: Use hardware authentication or always check the website domain name.

Example: If you intend to visit “douce-framboise.com”, make sure it’s not “douce-framb0ise.com”.

Forgotten Authentication Paths

Sometimes web service maintainers forget that the main login page isn’t the only access point.

Examples:

• Password reset via an email link may bypass two-factor authentication.
• Using external services via APIs (like Google or Facebook OAuth2 login) might also skip two-factor checks.

Always ensure two-factor authentication is enforced for all access paths to your users’ data.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

The password to rule them all

First of all, how can you create a good password you can remember?

Replacing characters is not good

Often, a bad but seemingly good idea pops up: replacing characters with others.

For example, using “p@ssw0rd” instead of “password”.

This idea comes from the English-speaking world and is based on the fact that most passwords were composed of ASCII characters, meaning the 26 letters of the Latin alphabet plus some punctuation symbols.

This article was originally written in French and was therefore aimed at French speakers who are less affected by this issue because French contains characters not included in the basic ASCII table. Accents, for example: “à, é, è, ù…” are included in the extended ASCII table or even UTF-8 (a table that contains alphabets from all over the world: Arabic, Cyrillic, Chinese…).

This technique makes passwords much harder to remember. Because they’re hard to memorize, we tend to make them shorter, which is also very bad.

Passphrases are good

This technique is known thanks to the xkcd comic strip, often referred to as “correct horse battery staple”.

To be effective, the combination of words must be as rare as possible. In the xkcd comic, this rarity is conveyed by randomness.

But let’s not forget that this is a password we need to remember. Our brains don’t handle randomness well. That’s why, at the end of the comic, the author emphasizes creating a mental image of the passphrase.

Note: In French, word frequency and word combinations are lower than in English because French speakers are fewer and the language contains articles, determiners, and possessives that further reduce combination probability. Thus, randomness is less critical than in English.

We can therefore use grammatically correct phrases as passwords.

An example to illustrate:

Which do you think is the better password:

“!He_r@spberr1es”

or

“The Queen of the Night’s Aria, The Magic Flute by Wolfgang Amadeus Mozart”

If you like Mozart, it’s definitely the second one. It’s easy to remember, longer than the first, and the chance that two people have the same password is low.

The password manager to bring them all

Having a password you can remember is good. But even better is being able to use passwords you cannot remember yourself. This reduces the risk of accidentally revealing them in conversation.

Jimmy Kimmel has a segment on his show available on YouTube.

To avoid this risk and create passwords you can’t remember, you need a password manager.

This is software or a web service that stores your passwords and displays them when requested. You just copy-paste them or use plugins that auto-fill login forms. They’re usually encrypted, and you access them with a master password. The only one you really need to remember.

With this system, since you don’t have to remember passwords, go big. Use 50-character randomly generated passwords with lots of weird symbols. Take advantage of UTF-8, checking compatibility with your software and services (not all support it, and some don’t display Cyrillic correctly, for example).

Software or web service

The advantage of using software instead of a web service is reduced risk of attacks and leaks.

A web-based password manager attracts attackers because a breach could expose thousands of accounts instead of just one.

There are also legal conflicts between the service’s jurisdiction and users’ rights, for example, European GDPR vs. the U.S. Cloud Act. We’ll discuss these legal aspects in future articles.

The advantage of a web service is a lower risk of password loss and a dedicated team monitoring and patching vulnerabilities.

Me

I use KeepassXC.

It’s open-source, meaning the code can be easily audited.

It runs locally on your devices instead of online, reducing centralized attack surfaces.

It encrypts passwords with the master password.

Legal conflicts between European GDPR and the U.S. Cloud Act are avoided since passwords aren’t stored on U.S. servers.

However, it’s harder to share the password database across devices. You also need to manage backups of your encrypted password database yourself.

KeepassXC is a variant of KeepassX (development appears to have stopped), itself a variant of Keepass, which is Windows-only. In fact, Keepass will undergo a security audit by the European Commission.

And in the darkness bind them (attacks)

Mass attacks

Mass attacks aim to collect as many passwords as possible, without caring whose accounts they belong to.

Dictionary attacks

The most common are dictionary attacks. They require little computing power but lots of storage (time-memory tradeoff). They test the most common known passwords (from previous breaches) and move on if they fail.

A yearly list of the 25 most popular passwords is published (here’s the 2018 list, with 2017 comparison):

• 123456 (unchanged)
• password (unchanged)
• 123456789 (+3 spots)
• 12345678 (-1)
• 12345 (unchanged)
• 111111 (new)
• 1234567 (+1)
• sunshine (new)
• qwerty (-5)
• iloveyou (unchanged)
• princess (new)
• admin (-1)
• welcome (-1)
• 666666 (new)
• abc123 (unchanged)
• football (-7)
• 123123 (unchanged)
• monkey (-5)
• 654321 (new)
• !@#$%^&* (new)
• charlie (new)
• aa123456 (new)
• donald (new)
• password1 (new)
• qwerty123 (new)

The issue with this list, present in all “how to make a good password” articles, is that it’s from the English-speaking world and doesn’t reflect the most common passwords in other languages (as french). If you have sources for other languages passwords, I’d love to see them.

Of course, there are also lists with 100, 1,000, 10,000, 1 million, or more passwords circulating online.

Rainbow tables

The second type is the rainbow table attack. It’s more computationally expensive but requires less storage. It only works if the attacker has obtained a database of password hashes (we’ll explain “hash” in a future article).

A rainbow table compares its stored hashes with those in the hacked database. If they match, the password is the same.

Example:

The attacker has a rainbow table with these md5 hashes:

• password: 5f4dcc3b5aa765d61d8327deb882cf99
• sdhfksdkfh: e2a1dbf388e7a7e87dc02943e3521036
• raspberry: 54a9fc48a5b664772e2ca06d1e0772d9

They hack a site’s hash table and get:

• Bob: 098f6bcd4621d373cade4e832627b4f6
• Alice: 54a9fc48a5b664772e2ca06d1e0772d9
• Lee: 190a16e29799b184d9900690ee04438a

The second hash matches the third entry in their rainbow table, so Alice’s password is “raspberry”.

Passphrases counter both these attacks. If the passphrase isn’t in the dictionary or rainbow table, the attack fails.

In a future article, we’ll discuss strengthening hash tables for website developers.

In the hash article, we’ll also see how to reduce rainbow table storage needs.

Brute force (grrrr)

The last technique is brute force: trying every possible combination. It’s the most computationally expensive but requires the least storage.

It’s the oldest method and why we started adding special characters to passwords.

But attackers have long used extended ASCII (with uppercase and accents) or UTF-8 (all alphabets worldwide: Arabic, Cyrillic, Chinese…) to test everything.

To counter it, use longer passphrases and the full alphabet (not “aaaaaaaaaaaaaaaaaaaaaa”).

This increases entropy (we’ll explain entropy in another article). French, with its accented letters, provides even higher entropy than English.

Example:

“!Es_fr@mbo1ses” has 72.1 bits of entropy, which is strong (good for most cases).

“The Queen of the Night’s Aria, The Magic Flute by Wolfgang Amadeus Mozart” has 475.8 bits of entropy, making it ultra-strong (obviously good).

The higher the entropy, the longer brute-force takes. The goal is to make attacks take years or centuries, not seconds. And with a password manager, take full advantage of UTF-8 (checking compatibility).

Targeted attack

A targeted attack aims to hack a specific user.

Keylogger

This involves infecting the user’s device with a tool that records every keystroke. When the user types their password, it’s captured.

This is outside this article’s scope and will be covered in future articles.

Social engineering

Through social engineering, a hacker may obtain your password from friends, family, or even you. Remember Jimmy Kimmel.

First, never share your password with anyone. Many services (including email) allow account sharing without sharing passwords.

Second, a password manager helps here too. If you don’t know your own password, a hacker can’t extract it this way.

For your master password, try a unique word combination or phrase no one knows.

Tools to find them

Yes, for those catching the title reference, I had to swap titles for coherence.

Once you have passwords for online services or devices, you must monitor them and track their lifecycle.

Make sure deleted passwords are truly deleted.

Expiration

You can set an expiration date for passwords. I don’t recommend this unless you’re sure because it often clashes with human behavior.

People quickly create slightly modified versions of old passwords or stop changing them altogether. Since password expiration handling isn’t standardized, this can create security flaws.

Breach monitoring

You should also ensure your password hasn’t been breached. Use monitoring tools for this.

A great service is haveibeenpwned by Troy Hunt. You can monitor your emails, and if they appear in a breach, the service notifies you.

It even provides tools to integrate into your website or app. However, I have serious GDPR concerns about it (to be discussed later).

Lastly, here’s a Twitter account showing how hard it is for services to handle passwords correctly:

https://twitter.com/PWTooStrong

Well, I’ve promised a lot of future articles. Great, more work for me. Pfff…

For developers, here are some tips on implementing passwords.

Note: This post was translated from french with the help of AI. The original post was written with the knowledge of a younger me.

Password Rules

If you’re tempted to set password rules like “between 8 and 20 characters” or “3 uppercase letters,” stop right there.

This approach is outdated, overly rigid for users, and leads them to create passwords that are hard for them to remember but very easy for bots to crack.

Don’t believe me? Check out the Twitter account passwordistoostrong.

Example:

Imagine the rule:

“Between 8 and 20 characters, at least 3 uppercase letters, and a punctuation mark.”

This password would qualify: “AbCdEfgh!” even though it’s not actually strong.

Meanwhile, the user is left frustrated.

Entropy

Instead, use an entropy-based system.

The zxcvbn library (named after the bottom row of keys on a U.S. keyboard) is a great implementation for calculating password strength based on entropy. It’s available in a wide variety of programming languages.

The advantage is that there are no rigid rules for estimating password strength. The library calculates the complexity of your password (considering the rarity of characters, their patterns, etc.) and assigns it a score. You can then decide from which score threshold you consider the password sufficiently secure.

Of course, you must use the same scoring and thresholds on both the server and client sides. If they differ, the server has the final say since the client side can be tampered with by an attacker (it’s not secure).

Avoid Compromised Passwords

Once you’ve set a threshold to block weak passwords, you still need to check if the entered password hasn’t already been compromised.

This is an optional additional safeguard because if your complexity threshold is high enough, the way the password is stored can already effectively defend against rainbow table attacks. Still, it’s a valuable bonus.

To do this, you can either load a JavaScript file on the client side or make an AJAX request to an API when the user creates their password.

If your AJAX request sends the plaintext password, that’s a problem. It increases your attack surface.

Loading a JavaScript file can be heavy, so it’s better to load it asynchronously. However, even asynchronously, the file size (and therefore the number of compromised passwords it can check) is limited.

Naturally, since the client side is not secure, you must repeat the verification on the server side.

Another technique is to use the Have I Been Pwned service. It offers a technically elegant solution but, as it’s an Australian service using U.S.-based infrastructure, it raises serious concerns about GDPR compliance in Europe (we’ll cover this in a future article).

This service works using k-anonymity.

You hash the user’s password and send only the first k characters of the hash to the service. The service returns all hashes in its database that share those k starting characters. You then compare them to the user’s password hash; if any matches, the password has been compromised.

Example:

The password “framboise” has this SHA256 hash:

“F9E4141EE43A3B877758E2584A1F6A0E7A9C8D6E58BB859A1665D8C1F447003C”

The first 5 characters are “F9E41.”

You send those 5 characters to Have I Been Pwned.

It returns:

• “F9E4141EE43A3B8777ERT8584A1F6A0E7A9C8D6E58BB859A1665D8C1F447003C”
• “F9E4141EE962758777ERT8584A1F6A0E7A9C8D6E58BB859A1665D8C1F447003C”
• “F9E4141EE43A3B877758E2584A1F6A0E7A9C8D6E58BB859A1665D8C1F447003C”

You compare and see that the third hash matches the user’s hash. Therefore, the password is compromised.

With this system, no password is ever transmitted over the network, and Have I Been Pwned never knows which password you’re testing. In the example, three results were returned, but the smaller the k value, the more results you get.

Then Store Passwords Properly

After the user has chosen a good password and securely submitted it, you must store it properly.

A fundamental rule: never store passwords in plain text. If an attacker gains access to your password storage, you’ve handed them access to every account on your service, and possibly other services (users unfortunately often reuse passwords).

So, what should you do?

1. First, limit password length, but not too much. Drupal, for example, limits password length to 512 bytes, meaning between 128 and 512 UTF-8 characters. This step is necessary to prevent a DoS attack because hashing consumes resources (an attacker could input an extremely long password to exhaust system resources during hashing).

2. Generate a random salt. A specific length string.

   

   You can generate a salt per database (for all passwords) or per password. It’s added to the password to ensure identical passwords have different hashes. Salts prevent hash comparisons, table-wide or row-wide, even if users choose the same password or reuse it across services.

   Example:

   Table-level salt:

   Alice uses “framboise” on both “nice-raspberries-squashed.com” and “mean-raspberries-squashed.com”.

   “nice-raspberries-squashed.com” generates salt “458,” making Alice’s password “458framboise.”

   “mean-raspberries-squashed.com” generates salt “gdf,” making Alice’s password “gdfframboise.”

   If a hacker accesses both tables, the hashes won’t match (since the salted passwords differ), so they won’t know Alice reused the same password.

   Row-level salt:

   Alice and Bob both use “framboise” on “nice-raspberries-squashed.com.”

   The site generates salt “458” for Alice’s password → “458framboise.”

   It generates salt “sdd” for Bob’s password → “sddframboise.”

   If an attacker accesses the table, they won’t know Alice and Bob used the same password because their hashes are different.

3. Hash the combined salt + password using a secure algorithm like SHA512.

   For password “framboise” with salt “458”:

   hash(sha256, ‘458framboise’) = ‘527D88733ED03CE5EF2D12AE4279950ABC29DC42817B14A58AB2150678A7CC72486CE637B8AD10333E80879F3D0CE685FA6CBB5DB8D7954382C2116616F8F6CF’

4. You now have a reasonably secure stored password. Also store the salt. To strengthen the hash further, you can iterate hashing with the previous hash and the password:

   hash(sha256, ‘527D88733ED03CE5EF2D12AE4279950ABC29DC42817B14A58AB2150678A7CC72486CE637B8AD10333E80879F3D0CE685FA6CBB5DB8D7954382C2116616F8F6CF framboise’) = ‘E3A252D3C455D638C82D8430DDC85E9B5B13F6508D690D97E2C5A1CC8AEE4A26D4C723E4EE4F524B4926C0357B2651132DF67446721BB4D3BCDD017C4A0E0E1B’

   The goal is to make the resulting hash different from any known hashes attackers might have, making password recovery harder.

   When a user tries to log in again, your system repeats the process (using the stored salt) and compares the new hash to the stored one. If they match, the password is correct.

Common Mistakes to Avoid

Sometimes we make silly mistakes! Here are some to avoid:

• When changing the complexity threshold for accepted passwords, don’t apply it to old password fields. Otherwise, users won’t be able to enter their old password to change it. Only apply the threshold to new password fields.

• If you change your password storage method or complexity rules and need user action, notify them by email. Don’t send password reset links with long expiration—users may click them months later, giving attackers time to exploit them.

• Never send a user’s password via email. Emails are like postcards: anyone can read them. And if you can email a password, it means you’re not storing it securely.

• Provide meaningful error messages when users create passwords. Use labels like “weak,” “strong,” or “very strong,” not entropy scores users won’t understand.

• Ensure passwords are transmitted via HTTPS. Ban HTTP completely.

• For extra security, implement authentication chains, meaning two-factor authentication (e.g., with a smartphone token) in addition to a password.

Remote git flow breaking

If you store a secret on git, you are supposed to store it encrypted (obviously).

To do a versioning you simply use a commit. As the encryption algorithm is non-deterministic (as it should be, it is not a hash), you end up with a blob of encrypted data which is entirely different from the previous commit. It makes impossible to do a compare and the full file is uploaded everytime.

Now, more funny. You want to do a git merge because 2 branches received separate updates on their secrets. As you have to compare 2 entirely different encrypted files, what advantages can offer any merge strategy from git. You definitely will end up screwing up your file and loose the contained secrets.

Local git flow breaking

You could say “this way I manage my secrets with the same flow as my code”. And I shall answer “Really ? Are you really sure on that ?”

So let’s have a look at your local flow, as I already explained that your remote flow is definitely dead.

• You pull the repo
• You integrate it into your VSCode or your terminal or any other IDE
• You change something on the code
• You commit
• You pull and push

Okay, now for the secrets.

• You pull the repo
• You integrate it into your VSCode or your terminal or any other IDE
• So far so good
• Hopefully you installed the tool to decrypt your secrets file
• You decrypt it (hoping the tool is integrated in your IDE)
• You change something on it
• You encrypt it with the same tool or another
• You double-triple check that you did not leave the unencrypted file somewhere on your workspace
• you commit with a drop on your forehead and a pain on your stomach
• you pull
• you try to figure out how to solve the conflict between your 2 blobs of encrypted file because a new commit on remote already altered the file
• you commit again hoping you did not screw everything up
• you push
• you see afterwards that you commit the unencrypted file as well and secrets are now freely available on the remote
• you desperately try to reset screwing up the git flow for the whole team
• you cry

I’m not sure it is the same git flow.

As conclusion, store your secrets in storages made for that. For instance, Hashicorp Vault on your infrastructure and keepassXC locally.

Attack surface

The access to the authentication panel to tape the administrator password is the only thing needed. This panel is available from any account on the device (locally or remotely via VNC protocol).

The flaw comes from the mistakenly activated root account on MacOS High Sierra. This root account is an inheritance of the UNIX system from which MacOS is developed. It is present and used in the Linux world but normally is deactivated (available but deactivated) on Mac, replaced with administrator accounts that cannot access system files.

How to protect yourself

You just need to set a password for this root account. Never loose this password.

In command lines from whatever administrator account on the device

• Open the terminal. The application is available in the folder Applications > Utilities.
• Enter the following command line then press Enter: sudo passwd -u root
• A password will be asked, enter the password of your root account then press Enter.
• Enter the same password to confirm and press Enter again.
• Your root account is configured with a password.

With the graphic interface

• Go to the menu > System Preferences.
• Click Users & Groups (or Accounts).
• Click the padlock, then enter an administrator name and password.
• Click Login Options.
• Click Join (or Edit).
• Click Open Directory Utility.
• Click the padlock in the Directory Utility window, then enter an administrator name and password.
• From the menu bar in Directory Utility, choose Edit > Change Root Password...
• Enter a root password when prompted.

SSL/TLS is a protocol that provides server identity verification (for instance a web server like Limawi, in that case we speak about https because the protocol to access a web server is called http). This identity verification works with a certificate sent from the server.

Here is the protocol step by step (technical terms are present in the graphs):

• The client software asks the server its identity
• The server sends a certificate signed by the certification authority trusted by both the client software and the server to the client software
• The client software checks if the signature belongs to the certification authority it trusts
• It sends a request to this certification authority to check if it ensures that the certificate is still valid
• The client software and the server agree on a session key that will encrypt informations during a limited period (this time passed, another session key will take the place, the details for Limawi are explained in the graphs about the session key)
• The client software and the server can communicate in a secure way

The HSTS extension

The HSTS extension is a http protocol extension (the protocol that loads webpages) reinforcing the SSL/TLS use.

This extension forces the browser to load secure versions of a webpage and all the ressources this page holds (that is to say https versions) in a domain (the base address of a website) that implements it.

If the wanted webpage does not have any secure version, it is not loaded on the browser.

Once the browser accesses a website implementing this extension it keeps in memory that the next webpages wanted on that website must be secured. It can check it even before sending the first request to the website.

A HSTS extension is valid for a limited period and the browser must check at the end of this time if the website always uses this extension to start another period.

Domains (base address of a website) can be preloaded in a database available in your browser. That way the browser knows even before sending the first request of its history to a webpage that it must be secure otherwise it does not load it.

This base is available here: HSTS Preload

With Limawi

With Limawi, the certification authority is Let’s Encrypt.

The session key exchange between the client software and the server is done with the Diffie-Helman protocol that creates a symmetrical session key without exchanging secret elements.

Limawi uses HSTS. The HSTS validity period is 6 months.

References

• SSL/TLS: Wikipedia
• HSTS: Wikipedia
• Diffie-Helman: Wikipedia
• Preloading for Limawi: HSTS Preload