A personal blog is, at its core, a handful of Markdown files and some HTML templates. Throw a bit of pandoc in the mix and you have a full fledged website.

I always wanted to write my own small blog engine. I want tiny features that no other blog engines have. For example why not adding a small game you can play:

This is a different kind of software. Not a product you maintain forever, but a small tool that does exactly what you need today and can be reshaped tomorrow at will with a bunch of prompts. You don’t have a product per se but a lot of markdown files being duct taped by open source projects. All managed with a coding agent acting as your obedient assistant.

What I ended up with

This site is now generated by a ~500-line shitty Python script I wrote with the help of Claude Code. It reads posts from a posts/ directory, runs them through Pandoc, renders Jinja2 templates, and writes static HTML to a dist/ folder that Cloudflare Pages serves directly.

Deployment is a git push. As it should be.

I must admit I’m using AI to detect which tags a post should have. I don’t think it’ll work perfectly, but the cost of changing it is small. And the icing on the cake wasn’t just that. It’s that I can use coding agents to actually interface with the blog’s code, so I can customize the UI to my own needs.

I also used AI to migrate the old posts to plain markdown stored in a git repo. The CSS migration is not perfect, so some old posts will still render like shit, but I have plenty of time to fix them.

Harnesses are not only for code

Something I want to experiment with is the use of pi as an assistant on working in the blog. You can reshape pi to do whatever you want. In the blog repo, there is a small pi UI component that shows how many posts and drafts I have. I use VS Code to write and pi to proofread, correct typos, handle assets, and change the build…

Everything is just text

At the end of the day, if you have a sufficiently powerful harness you can shape your output to your will. It is a weird feeling. I am used to the safety of deterministic software (or at least the illusion of it), and embracing this “lack of knowing everything” is a bit worrisome.

Anyway, the future is bright. I can add a small game to a blog post just to prove a point. The future is surely weird.

Have fun!!

But I kept seeing people push further — giving agents a longer leash, taming them into handling real features end to end. And it was working. The gap between what I was doing and what seemed possible was growing.

So I started trying. And the results were mixed. Sometimes the agent would nail it. Other times it would go in circles, making reasonable-looking choices that were completely wrong for the project. Wrong abstraction. Wrong assumptions about the data model.

The difference, I've come to believe, is context. Not prompt engineering tricks, not model selection. Context — how much of it you provide, and how you provide it.

The colleague problem

When I sit down with a colleague to discuss a feature, we share a tremendous amount of context without even thinking about it. We both know the codebase. We both know the users. We both know the last three decisions that shaped the architecture. We both know which parts of the system are fragile and which are rock solid.

An agent knows none of this. It starts every conversation from zero. And yet I routinely catch myself typing something like "this thing is happening and it shouldn't" and expecting the agent to make the same choices I would.

It won't. It can't. Not because it's incapable, but because it doesn't have the information it needs.

What actually helps

I've started paying attention to what makes a prompt produce good results versus mediocre ones. It's not about writing more. It's about including the right things:

• What I'm building and why. The why matters more than I expected. "Add a login page" and "add a login page because we need to gate access to paid features and our users are non-technical" lead to very different implementations.
• Constraints. What libraries am I already using? What patterns does the codebase follow? What can't change? Normally I provide those in the form of documentation od the project.
• Decisions already made. If I've already decided on JWT over sessions, I say so. Otherwise the agent might pick the other one and I'll waste twenty minutes unwinding it.
• What I'm unsure about. This is the one I used to skip. But telling the agent where my thinking is still fuzzy turned out to be incredibly valuable. It fills gaps I didn't know I had.
• How to know when it's done. This one changed everything for me. If you can tell the agent "run this test suite" or "check that the API returns 200 on this endpoint," you've closed the loop. The agent can try something, verify it, and try again if it didn't work. Without a definition of done, it generates code and hopes. With one, it iterates.

And that last point is where things start to feel different. When the agent can check its own work, it stops feeling like a text generator and starts feeling like a harness — like you're constraining the execution of a wild creature that can produce thousands of lines of code per second, and your job is to point it in the right direction and let it run.

The agent as a thinking partner

The shift that changed my workflow the most was to stop treating the agent as a code generator and start treating it as a thinking partner.

I used to try to write a complete spec before handing anything to the agent. That felt responsible. But it was often slower and worse than writing a rough one and refining it together. The agent spots gaps in my thinking. It asks about edge cases I haven't considered. It surfaces assumptions I didn't know I was making.

My workflow used to be: think hard, write complete spec, hand it to the agent, get code back. Now it's more like: write down what I know, let the agent challenge and expand it, arrive at a better spec together, then build from that shared understanding.

The second approach produces better specs and better code, because the agent understands the reasoning behind the decisions, not just the decisions themselves.

Building this into the process

I liked this idea enough that I built it into a reusable skill I call interview-spec. You point it at a rough specification file, and instead of immediately generating code, it interviews you.

interview-spec on GitHub Gist

First, it reads the spec and identifies what's there and, crucially, what's missing. It looks for gaps in error handling, edge cases, security considerations, performance implications, and operational concerns.

Then it starts asking questions. Not obvious ones. It asks about race conditions in the data model. About what happens when the network fails halfway through a transaction. About how to roll this back if something goes wrong in production. About which parts are truly MVP and which are nice-to-haves that got unconsciously promoted to requirements.

It asks 2-4 questions at a time, drills deeper based on answers, and periodically summarizes what it has learned to make sure we're on the same page.

By the end, it updates the original spec with everything that was uncovered and produces a prioritized task breakdown with complexity estimates and dependencies.

The key insight for me was that this back-and-forth is not overhead. It's the actual work. The conversation is the specification process. Every question the agent asks is a question that would have surfaced eventually — probably at a much more expensive time, like during implementation or, worse, in production.

Making context persistent

This goes beyond individual conversations. I've started making context available in places the agent can find on its own:

• A AGENTS.md (which I symlink to CLAUDE.md) file in each repository root with project conventions, architecture decisions, and patterns to follow.
• Clear commit messages and PR descriptions. The agent reads those too.
• Documentation that captures why things are the way they are. Not exhaustive docs — just the decisions that matter.

Every piece of context I make available is a decision the agent doesn't have to guess about.

Letting the agent find its own context

There's another dimension to this that took me a while to appreciate: you don't have to hand the agent all the context yourself. You can let it go get it.

Tools are how an agent builds its own context. Every tool call is the agent reaching out into the world to learn something it didn't know. Reading a file, running a command, hitting an API — these aren't just actions, they're the agent filling in its own gaps.

And the interesting part is that you don't always need formal tool integrations for this. Sometimes just reminding the agent what's available on the system is enough. If gh is installed and configured, telling the agent it can use it means it can traverse a PR, read its comments, check CI status, and piece together the full story behind a bug — all without me having to copy-paste any of that into the prompt.

I've started thinking about it this way: I provide the why and the constraints, and I make sure the agent has the tools to discover the what on its own. That split turns out to be a much better use of everyone's time than trying to front-load every detail myself.

The real cost

Providing good context takes effort. It forces me to articulate things that feel obvious and to confront the parts of my plan that are still vague. I can't hide behind ambiguity anymore.

But that effort isn't wasted. It's the same effort I'd spend debugging a misguided implementation, rewriting code that went in the wrong direction, or wondering why the agent "did it wrong" when the real problem was that I never said what "right" looked like.

Context is not a tax on using an agent. Context is the work. The agent just makes that truth impossible to ignore.

The game is Don Quijote. A text based adventure.

Yes. Just some super naive graphics. A small REPL-like interface and you are good to go to live amazing adventures.

Since we are in the world of AI one tiny fun experiment I fantasized about is letting the AI play the game on its own to see how far it could go.

Even if I loved that one it is not the optimal choice to use as a learning point. I want something with better feedback on the status of the game.

There is one oldie but goldie I can use for that.

Well, I can use many games in reality but is a good one because it has a score. In the game you have to get some items, place them in the shelves and you get points for that.

Experiment 1: The naive start

Piece of cake. Just start your application, and have a python script remote control that window typing the contents.

This is a tiny example so GPT-5-mini is a nice start.

Since we are talking about on screen text the starting point can be just OCRing with something like tesseract (so it is faster) and send the things to the llm outputting the response.

First gameplay? Surprisingly fine.

Ok, ok. I know going rogue on that troll was not the smartest thing. But for a first iteration it is not bad. Now, unfortunately it is very hard for the LLM to know that it can resurrect or even restart the game. So let's add some small context.

Experiment 2: Giving more context to the agent

The LLM, at this moment is treating each and every room individually as it saw it afresh. In Zork there is this weird Loud Room that just echoes everything you type. The agent gets crazy and tries to restart the game. It is fun because that is one room where you cannot restart the game. That makes it enter an infinite loop.

A naive solution for this is to just turn the dialog of the game in a conversation with the agent with the agent storing all the messages in the context.

For long sessions the context may rot so let's implement a sliding window. I experimented a bit with the number of messages to keep. I started with 50 or 100. With those numbers the game plays far better. Anyway, there is not enough context to learn from more than one game. You just avoid the last death in the best case scenario.

Experiment 3: Giving it long term memory

Right now, the agent learns a bit from history but it would be great if it had the ability to keep things like where objects are and what happens if you try to swing your sword to that troll.

That is the purpose of the long term memory. You distill the content a bit more there to avoid having a lot of pollution and you also keep a short term memory so recent events have some degree of preeminence.

In LangGraph we have the checkpointers that allow us to do this easily.

from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.sqlite import SqliteSaver
from langchain_anthropic import ChatAnthropic
from typing import TypedDict, Annotated
from langgraph.graph.message import add_messages

class State(TypedDict):
    messages: Annotated[list, add_messages]

def chatbot(state: State):
    llm = ChatAnthropic(model="claude-sonnet-4-20250514")
    response = llm.invoke(state["messages"])
    return {"messages": [response]}

graph = StateGraph(State)
graph.add_node("chatbot", chatbot)
graph.add_edge(START, "chatbot")
graph.add_edge("chatbot", END)

checkpointer = SqliteSaver.from_conn_string("chatbot_memory.db")
app = graph.compile(checkpointer=checkpointer)

config = {"configurable": {"thread_id": "conversation-X"}}

That will persist all the messages states in sqlite. It is not the best way to do it but for now it will suffice.

In our case we will use the game as a thread_id. This way the conversations with the agent won't be mixed between games.

But probably you will be thinking... we are doing the same thing we were doing before, aren't we? And yes, you'd be absolutely right. With this we just implemented short term memory. We need to store some kind of long term memory too.

We will introduce the concept of a game_guide in the state of the agent

class AgentState(TypedDict):
    messages: Annotated[List[BaseMessage], operator.add]
    last_observation: str
    action: str
    game_guide: str
    steps: int

So the state of the agent now has a game_guide that is updated every X steps. It is like the agent is writing one of those gamefaqs.com guides in text and now that becomes our source of truth.

With this the game gets pretty far. Sometimes it still gets stuck. Especially in areas where the map is special like the forest.

Even if you have the history the model tends to take "impulsive" decisions. If there is a mailbox tends to overspend attention on it taking a look to it and overplaying with it when there is probably nothing else to do.

This is a bit similar to what we do in Claude Code when we compact the history.

Results

I got to 40 out of 350 points.

Impressive IMHO. To be honest I could have got more but I did not want to spend more than 5 bucks in openAI.

Lessons

The prompting is super important. When writing the game guide, it took me some time to notice that it was including information about the state of the current run instead of generic information about the game and that was confusing to subsequent runs.

Observability is a fun thing to do. I keep the plain text file for the memory so I can see what the AI beliefs are at the moment. I am missing a lot of info, though.

Will I continue? Obviously. I really think I can get this to win the game somehow. The cost is a problem now. Probably I will set up something using ollama or some local, even though slower, llm.

The adventure awaits

This post on how to workflow image generation with nano banana is amazing. It is giving me a lot of inspiration for some tooling I want to create for my children.

Alcohol consumption is decreasing but there are unintended consequences. https://www.freddyvega.com/p/alcohol

I read a post in Reddit about a desperation score and some dark patterns some companies used. To be honest, having worked in the food delivery industry that sounded like an absurd thing to me and it finally happened to be a hoax. It was fun to read how the scammer used AI to fake his evidence... Come on, a company using latex for its documentation?

I've been using this form of interviewing lately to refine specs. It is a nice way of using AskUserQuestion. Another way of using claude as a rubber duck.

I have a lot of fatigue on the concept of personal productivity. This piece summarizes a bit how I tend to feel about it.

My biggest use case is that I normally want to break the work I've done into more than one commit and I'd like to see how everything is looking like meanwhile I work.

It just happened that there is a bug in gitui that happened to be a problem for me. I am using bitwarden as ssh-agent (not 100% happy with that yet) and I neet to manually specify which key to use for some sites. Then opportunity appeared.

Why not using the same git TUI for the shell and for Neovim?

I settled on Neogit since it looks like my good old beloved Magit. Then starting a new neogit session is as easy as:

alias g = nvim -c "lua require('neogit').open()" -c "autocmd TabClosed * if tabpagenr('$') == 1 | quit | endif"

The alias also sets a callback so that, if I press 'q' I go back to the shell automatically

This way I am using the same tool in when I am using the shell and when I am coding using neovim. Cool!

The latest mastodon update brought quotes). I'm super happy to see mastodon community advancing in a healthy state.

Google releases MCP for all its services. I need some time to experiment with this but I hope I can do nasty things with drive.

Skills are now an open standard). It is cool to see some open standards to be proliferating on this new AI world. I am still concerned on how portable we should be from one LLM client to another (spoiler alert, it should be 100% portable with no friction).

Questionable ethics aside Anna's blog is always super interesting to read for technical details. This post on their efforts to backup Spotify is no exception.

I've been playing with text based adventures recently (blog post incoming), but it looks like there is smarter people than me doing it already.

Even if you don’t train models, walking through the llm evaluation guide will give you a lot of ideas. It is amazing that this kind of content is still free.

Today we have a portable way to do that between llm clients: the skills. Skills are able to do something similar to this in llm clients. We only have to win having an open standard for this.

I will finally take my markdown folders and convert them to this format.

This will, for sure be a popular way to automate internal processes.

The real chameleons...

I am finding myself lately using more and more GUI tools. Maybe because in my day job I am using more MacOS but probably because the nature of the things I am trying to do is changing a little bit. I used to rely a lot in saving terminal sessions but these days I am using more OBS or Gifox than ever. Anyway I always find comforting having tools like vhs available. They give peace of mind.

Another nice episode of The cyberpunk librarian. Talking about personal freedom and self hosting.

This week I went through the first chapter of Casella and Berger. I am supposed to know those terms but it is funny how reading a technical text makes you settle all your concepts in such a way that unless you are not 100% clear in the concepts it is super hard to follow. I guess I will spend a couple of weeks in the exercises...

This interview in Freakonomics with Dan Wang about his book Breakneck has an amazing metaphor I will use more. He talks about lawyer mindset vs engineer mindset. The engineer has problems and tries to solve them. What he calls “the lawyer mindset” is that you are thinking in the process of solving the problem instead. Defining a good process becomes more important and hurdles are found for the people actually doing the work.

I, briefly, tried Antigravity and I was far from impressed. After watching this video I am starting to question myself.

The Non-Negotiables

First, the good news: many of my essential tools (Neovim, VSCode, etc.) are cross-platform, so they came along for the ride. But some aspects of my workflow needed Mac-specific alternatives. Let's dive into what I'm using to make macOS feel more like home.

Without further ado, my picks:

AeroSpace: The i3-like Tiling Window Manager

GitHub - nikitabobko/AeroSpace

The biggest gap to fill was a proper tiling window manager. AeroSpace has become my go-to solution. While it's not as configurable as XMonad, it brings several i3-like features that make it a solid alternative.

Aerospace does something insanely well. Something that grinds my gears in mac is that the animations on workspaces take about 500ms which is a huge waste of time. This completely ignores macos workspaces with its own implementation. Huge success.

Sort of scratchpads

Scratchpads are probably the feature I liked the most in XMonad. You can anyhow create a back and forth workspace so you can have something similar. Not perfect but good enough.

cmd-o = 'workspace --auto-back-and-forth O'

[[on-window-detected]]
if.app-id = 'md.obsidian'
run = 'move-node-to-workspace O'

With this I can have press cmd-o to go to obsidian workspace and pressing cmd-o again takes you back.

Keyboard driven window focus

And then set some shortcuts to focus common apps.

To do that I created a ~/.config/aerospace/focus.sh script that does:

#!/usr/bin/env bash

APP_NAME=$1

app_window_id=$(aerospace list-windows --all --format "%{window-id}%{right-padding} | %{app-name}" | grep $APP_NAME | cut -d' ' -f1 | sed '1p;d')
aerospace focus --window-id $app_window_id
aerospace move-mouse window-lazy-center

With that I can have the shortcut not to only focus on the app I want but move the mouse cursor to the middle of the window so I can deduct where the mouse is faster. I just have to add to the aerospace.toml the following

cmd-y = 'exec-and-forget ~/.config/aerospace/focus.sh Code'
cmd-u = 'exec-and-forget ~/.config/aerospace/focus.sh kitty'
cmd-i = 'exec-and-forget ~/.config/aerospace/focus.sh Firefox'

Small tweaks

I had to force Firefox's picture in picture to float

[[on-window-detected]]
if.app-id = 'org.mozilla.firefox'
if.window-title-regex-substring = 'Picture-in-Picture'
run = 'layout floating'

And then set some shortcuts to focus common apps.

Ice

GitHub - jordanbaird/Ice

I've used mac in the past and you end up with 9000 different menubar items you don't care about so using ice makes it easy for you to not clutter your menubar.

Alfred

Alfred - Productivity App for macOS

There are more modern options but I had a perpetual license for this since forever. I don't want another subscription for this so Raycast was out of the question. If you combine with stuff like https://github.com/yohasebe/openai-chat-api-workflow you will have more than enough (text, audio and vision) with paying just for API usage.

Notable alternatives

There are some software I don't want to avoid to mention but in the end it did not make the cut.

apptivate

Apptivate

To recover from the lack of scratchpads this was my first option. Anyhow it looks insanely outdated. It has some cool options, like warning you if you are overriding a system shortcut.

yabai

GitHub - koekeishiya/yabai

Yabai supports proper scratchpads but I have not been able to set up workspaces that work instantly and I'd have to disable SIP. Anyway, it is super cool and maybe some of you find it useful.

Hammerspoon

Hammerspoon

A swiss army knife for automation. You can assign lua functions to events. I tried to use it to have keybindings to focus to apps like:

-- Function to focus on app or launch it if it's not running
function focusOrLaunch(appName, bundleID)
    local app = hs.application.get(appName)
    if app then
        if app:isFrontmost() then
            -- If the app is already focused, do nothing
            return
        else
            -- Focus on the app
            app:activate(true)
            app:unhide()
        end
    else
        -- Launch the app if it's not running
        hs.application.launchOrFocusByBundleID(bundleID)
    end
end

-- Define keyboard shortcuts
-- VSCode
hs.hotkey.bind({"cmd"}, "y", function()
    focusOrLaunch("Code", "com.microsoft.VSCode")
end)

-- Kitty
hs.hotkey.bind({"cmd"}, "u", function()
    focusOrLaunch("kitty", "net.kovidgoyal.kitty")
end)

-- Firefox
hs.hotkey.bind({"cmd"}, "i", function()
    focusOrLaunch("Firefox", "org.mozilla.firefox")
end)

-- Obsidian
hs.hotkey.bind({"cmd"}, "o", function()
    focusOrLaunch("Obsidian", "md.obsidian")
end)

But I ended up leveraging aerospace for this so one less software to run.

The veredict

While it's not exactly Arch Linux, this setup has made macOS feel surprisingly comfortable. I hope I gave you some ideas on this post on stuff you can do to feel a little bit better using your good old MacOS computer.

Have fun!

I like data. I love data. Data is cool and necessary. And toxic if used wrongly.

"Metrics are fine so everything is fine". I am tired of seeing organizations and people to rely on metrics to do the job for them.

Metrics are becoming the opium of management. We forgot why we do stuff and instead focus on creating the feeling that everything is right.

I saw a message in a management message board saying that we have to move some people's ratings to another value so all values fit a bell curve. That is wrong in so many levels that I cannot deal with it. I have to write a blog post just in case I can help training whatever AI so less people act as dumb-asses.

Let's deal with this example.

In a healthy, big enough, organization performance ratings of the employees should be distributed on a bell curve. That means that everyone is getting enough of a challenge and statistics should do the rest.

You will have low performers for many reasons. I expect a person in a new role to perform poorly by definition. It makes sense and no one needs to worry about that. Also, everyone blips from time to time. We are all humans. Some cases are not reasonable and in a healthy organization we should not be scared of talking about them.

Not it comes the toxic part. When an organization tries to artificially fit their ratings to this curve for whatever reasons they are playing with the stats. They are losing every single reason you have for the bell curve to exist.

A wrong bell curve is not fixed by artificially making people stick to it. If there are some toxic elements in culture that make an organization not to stick to a bell curve they should be seen, known and communicated with big neon signs.

A wrong bell curve is a consequence, not a symptom.

Normally you compare departments and you expect all of them to share the same shape. That means every person is as fair as you can and everyone shares a common view of what a given rating is.

You don't assign quotas of ratings to people. That is insanely absurd and even more toxic than a wrong curve. You are putting incentives to deceive the system and just don't do the right thing but the thing that looks good instead. Let's put everyone in the middle so everyone has a happy face! You are creating a full management layer that don't have to take decisions rendering them useless.

Also, if you assign quotas you don't detect abnormal situations WHICH IS THE PURPOSE OF DOING THIS BLOODY EXERCISE.

Bell curves should appear with no "manual intervention". You should not be tricking the numbers. If everyone in the organization rates the same way they will appear and if they don't then you need to train people so all share the same view. You use the metric to be informed when you solved the problem, to detect places where you can act and not as a substitute for problem solving.

That is why I tend to use data informed instead of data driven. You want data as signals that you are right on track but they are not the goal itself. The map is not the territory.

At 4am evilmeow.com and all my sites (including this log started alerting)

Ping failing so let's see.

A restart "fixed" the problem for a couple of minutes and then it got down again.

The logs for some reason were not being sent to Loki. So we redirected dmesg to a file. Checking that file we saw the error message:

> [170902.115198] watchdog: BUG: soft lockup - CPU#0 stuck for 7342s! [CPU
> 1/KVM:1382]
>
> [170986.118899]  ? __hrtimer_run_queues+0x105/0x280
> [170986.118986]  ? clockevents_program_event+0xb3/0x140
> [170986.119112]  ? hrtimer_interrupt+0xf6/0x250
> [170986.119199]  ? __sysvec_apic_timer_interrupt+0x4e/0x150
> [170986.119288]  ? sysvec_apic_timer_interrupt+0x8d/0xd0
> [170986.119378]  </IRQ>
> [170986.119460]  <TASK>
> [170986.119543]  ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
> [170986.119634]  ? smp_call_function_single+0x11e/0x160
> [170986.119727]  ? __pfx___loaded_vmcs_clear+0x10/0x10 [kvm_intel]
> [170986.119840]  ? update_load_avg+0x82/0x830
> [170986.119928]  vmx_vcpu_load_vmcs+0x143/0x440 [kvm_intel]
> [170986.120037]  vmx_vcpu_load+0x19/0x60 [kvm_intel]
> [170986.120145]  kvm_arch_vcpu_load+0x48/0x240 [kvm]
> [170986.120320]  ? load_fixmap_gdt+0x44/0x80
> [170986.120407]  ? __perf_event_task_sched_in+0x88/0x200
> [170986.120497]  kvm_sched_in+0x3a/0x50 [kvm]
> [170986.120649]  finish_task_switch.isra.0+0x212/0x310
> [170986.120738]  __schedule+0x409/0x15e0
> [170986.120825]  schedule+0x33/0x110
> [170986.120910]  kvm_vcpu_block+0x58/0xc0 [kvm]
> [170986.121094]  kvm_vcpu_halt+0xde/0x460 [kvm]
> [170986.121246]  kvm_arch_vcpu_ioctl_run+0xab4/0x1760 [kvm]
> [170986.121409]  kvm_vcpu_ioctl+0x297/0x800 [kvm]
> [170986.121591]  ? kvm_on_user_return+0x78/0xd0 [kvm]
> [170986.121749]  ? fire_user_return_notifiers+0x37/0x80
> [170986.121842]  ? syscall_exit_to_user_mode+0x86/0x260
> [170986.121931]  __x64_sys_ioctl+0xa0/0xf0
> [170986.122023]  x64_sys_call+0xa68/0x24b0
> [170986.122132]  do_syscall_64+0x81/0x170
> [170986.122241]  ? do_syscall_64+0x8d/0x170
> [170986.122328]  ? do_syscall_64+0x8d/0x170
> [170986.122414]  ? do_syscall_64+0x8d/0x170
> [170986.122500]  ? do_syscall_64+0x8d/0x170
> [170986.122586]  ? do_syscall_64+0x8d/0x170
> [170986.122673]  entry_SYSCALL_64_after_hwframe+0x78/0x80
> [170986.122761] RIP: 0033:0x78d9dcb1cc5b
> [170986.122866] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00
> 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0
> ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
> [170986.123004] RSP: 002b:000078d9ca3fb0b0 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000010
> [170986.123109] RAX: ffffffffffffffda RBX: 000058fa7d6bf4a0 RCX: 000078d9dcb1cc5b
> [170986.123213] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000001d
> [170986.123318] RBP: 000000000000ae80 R08: 000058fa7b0f2c90 R09: 0000000000000000
> [170986.123422] R10: 0000000000000002 R11: 0000000000000246 R12: 0000000000000000
> [170986.123526] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000000
> [170986.123633]  </TASK>
> [171000.004671] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> [171000.004778] rcu:  7-...0: (0 ticks this GP) idle=1224/1/0x4000000000000000
> softirq=2170915/2170915 fqs=3901860
> [171000.004894] rcu:          hardirqs   softirqs   csw/system
> [171000.004982] rcu:  number:        0          0            0
> [171000.005072] rcu:  cputime:        0          0            0   ==> 7950223(ms)
> [171000.005176] rcu:  (detected by 0, t=7980227 jiffies, g=5655105, q=231104
> ncpus=8)
> [171000.005283] Sending NMI from CPU 0 to CPUs 7:
> [171000.005373] NMI backtrace for cpu 7
> [171000.005375] CPU: 7 PID: 391169 Comm: kworker/7:0 Tainted: G      D W I  L   
> 6.8.4-3-pve #1
> [171000.005376] Hardware name: FUJITSU D3417-B1/D3417-B1, BIOS V5.0.0.11
> R1.29.0.SR.1 for D3417-B1x               05/13/2020
> [171000.005378] Workqueue: md submit_flushes
> [171000.005383] RIP: 0010:native_queued_spin_lock_slowpath+0x7f/0x2d0
> [171000.005386] Code: 00 00 f0 0f ba 2b 08 0f 92 c2 8b 03 0f b6 d2 c1 e2 08 30 e4
> 09 d0 3d ff 00 00 00 77 5f 85 c0 74 10 0f b6 03 84 c0 74 09 f3 90 <0f> b6 03 84 c0
> 75 f7 b8 01 00 00 00 66 89 03 5b 41 5c 41 5d 41 5e

After reaching Hetzner support looks like there is a hardware failure and they will proceed to replace the full machine.

They provide access to the new machine and I proceed to restoring latest backup from Proxmox Backup Server.

I started with OPNSense. Backup was restored in 5 mins but the machine did not start.

The interface vmbr1 was off. I lost 1h debugging that. By default Debian's installation of Proxmox does not include the package openvswitch-switch and looks like I installed it manually and forgot to add it to the automations 🤦‍♂️.

Once the interface was in place everything started smooth as butter.… until 4 hours later.

The machine started failing again with a different dmesg error

> [31719.674017] BUG: kernel NULL pointer dereference, address: 0000000000000008
> [31719.674118] #PF: supervisor write access in kernel mode
> [31719.674197] #PF: error_code(0x0002) - not-present page
> [31719.674305] PGD 0 P4D 0
> [31719.674381] Oops: 0002 [#1] PREEMPT SMP PTI
> [31719.674460] CPU: 0 PID: 1180 Comm: kvm Tainted: G          I        6.8.4-3-pve
> #1
> [31719.674568] Hardware name: FUJITSU D3417-B1/D3417-B1, BIOS V5.0.0.11
> R1.29.0.SR.1 for D3417-B1x               05/13/2020
> [31719.674669] RIP: 0010:blk_flush_complete_seq+0x291/0x2d0
> [31719.674775] Code: 0f b6 f6 49 8d 56 01 49 c1 e6 04 4d 01 ee 48 c1 e2 04 49 8b
> 4e 10 4c 01 ea 48 39 ca 74 2b 48 8b 4b 50 48 8b 7b 48 48 8d 73 48 <48> 89 4f 08 48
> 89 39 49 8b 4e 18 49 89 76 18 48 89 53 48 48 89 4b
> [31719.674904] RSP: 0018:ffffaea0c936ba60 EFLAGS: 00010046
> [31719.674984] RAX: 0000000000000000 RBX: ffff9808d4040a00 RCX: ffff9808d4040a48
> [31719.675067] RDX: ffff9808d202b420 RSI: ffff9808d4040a48 RDI: 0000000000000000
> [31719.675149] RBP: ffffaea0c936baa0 R08: 0000000000000000 R09: 0000000000000000
> [31719.675231] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000029801
> [31719.675366] R13: ffff9808d202b400 R14: ffff9808d202b410 R15: ffff9808d3e97300
> [31719.675449] FS:  000073172a4006c0(0000) GS:ffff9817ae200000(0000)
> knlGS:0000000000000000
> [31719.675546] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [31719.675626] CR2: 0000000000000008 CR3: 000000010af80002 CR4: 00000000003726f0
> [31719.675708] Call Trace:
> [31719.675808]  <TASK>

This time anyway it looks like it is not a hardware issue accorting to Hetzner support. I rolled back the kernel to a previous version and it has been running flawlessly for a day.

Takeaways

• Every small thing you don't automate bites you in the ass. Even if it is just installing a package
• I am a bit concerned on the stability on the kernel in Hetzner machines. I will keep two of them installed in case I have to roll back.
• Having decent logging aggregation saved me again.

So, some more learnings in exchange for some downtime. Anyway I am relatively happy now and sure that in the case of catastrophic event I won't lose all the data. Another step in the path to making evilmeow.com open to the public.

I am not that good at mathematical abstractions so in order to understand something I prefer to take the opposite path and then see the reason of it. Why is it useful? Why all the fuss?

A Fourier transform is a way to decompose a signal into its components. If we have a signal with frequency of 5Hz and another with 50Hz combined I will have something like

import numpy as np
import matplotlib.pyplot as plt
from scipy.fftpack import fft, fftfreq

def signal(t):
    return np.sin(2 * np.pi * 5 * t) + 0.5 * np.sin(2 * np.pi * 50 * t)

sampling_rate = 1000  
T = 1.0 / sampling_rate  
t = np.arange(0, 1.0, T)  
y = signal(t)  

Y = fft(y)
N = len(t)
freqs = fftfreq(N, T)

Let's draw this signal

plt.figure(figsize=(14, 6))
plt.plot(t, y)
plt.title('Domain of time')
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
plt.grid()

You can see the big wave to be the 5Hz signal and the much faster wave the 50Hz signal.

If I apply Fourier transform to that and plot the absolute value of the result I will have something like.

plt.figure(figsize=(14, 6))
plt.plot(freqs[freqs>0], np.abs(Y)[freqs>0])
plt.title('Domain of frequency')
plt.xlabel('Frecuency (Hz)')
plt.ylabel('Magnitude')
plt.grid()

plt.show()

There are two peaks. One at 5 and another at 50. If we draw with stems it will be more clear

Exactly the ones we generated the wave from. But... still... why is this useful?

There are two things that I didn't explain yet:

• We can also do this for sampled signals, not only for continuous ones
• This is a two way transform

Let's start with the former. I can do Fourier transform on, for example a wav file. A wav file are sound wave samples at a given frequency (normally 44000 or 48000 Hz) and decompose them. We can do the same with color waves in an image or even whatever we think can behave as a signal. That makes the potential applications endless.

This being a two way transform is the final piece of the puzzle. I can go back and forth from Fourier transform to original signal. This allows us to manipulate components of the signal with ease and get back to the original.

In our previous example let's imagine that we consider the 50Hz signal (the one that vibrates more) to be noise. Let's do a small noise gate and filter that signal in the frequency domain

Y_filtered = Y.copy()
Y_filtered[50] = 0
Y_filtered[-50] = 0

This filtered signal will show as

So if we do the inverse Fourier over it

from scipy.fftpack import ifft

filtered_y = ifft(Y_filtered)

We get

We did something that can be improved, though, in the lines

Y_filtered[50] = 0
Y_filtered[-50] = 0

In out code we have that array called freqs. Since the array indexes are discrete values and a signal is a continuum we have this kind of arrays so we can know what the indexes are. In reality that 50 we want to filter is not an exact point but we have do do a small range. Let's say we want to do 0.1 wide.

Y_filtered[np.abs(freqs - 50) < 0.1] = 0
Y_filtered[np.abs(freqs + 50) < 0.1] = 0

That will set to zero all values that are at less than 0.1 to the 50Hz point. From 49999 to 50001. Probably we will have just one in our array but depending on the resolution you may have more.

So now we should have some super basic intuition about how this works, why is important and how we can use it.

Happy hacking!

First and foremost, do not forget what I want my homelab for. My homelab is excessive, overkill, baroque... Do I need a Kubernetes cluster? No, obviously not. Do I learn a lot by having and maintaining it? Hell, yes!

My homelab is optimized for learning. I try stuff, I am wild, I am naive and crazy. Sorry, not sorry.

Obviously it can be done better. I expect no less.

The sections are in no particular order.

🔌Hardware {#%F0%9F%94%8Chardware}

I have a small Thinkcentre m90q kubernetes cluster. It is 3 machines. All are a tiny bit different but they work fine together anyway.

Next to that I have my good old Qnap with 4x8TB disks in RAID5.

I 3d printed a new homebrew NAS. I want probably to ditch RAID for it. RAID is not a backup. I learnt that the wrong way. RAID only provides availability and just some extra reliability. I have to do the backups anyway and not having the data available in NAS for some hours is something I can definitely live with. Specially since I am moving the data that needs to be available to Longhorn volumes in Kubernetes. More on this later.

The new NAS is the fourth Proxmox node you see in the screenshot.

This one will also double down as Proxmox Backup Server.

☎️ Networking {#%E2%98%8E%EF%B8%8F-networking}

I moved production to Hetzner but I don’t want to use straight their platform. Maybe I am a bit cheap but I rented a Hetzner server and installed OPNSense on it.

Since I took that path in my servers I bought a small pfSense box and installed OPNSense on it. I use it to create a small Wireguard network so I can see my production virtual machines locally to ease debugging.

This means that at home I can do ssh myuser@192.168.2.123 and I will see the machine as if it was here. This is a bit more nuisance to set up at router level but it simplifies a lot the configuration of the continuous integration and deployment. I used to have VPN at computer level but setting this in my router makes everything far easier.

Security is a concern anyway. I know now my home becomes a gigantic security issue 🥲

The future

I use Wireguard tunnels to access my network from the outside but I want to start giving access to some friends. I will probably have to fight somehow for the IP address spaces and I don't know how to properly solve that yet.

Operating systems

I rely on Proxmox and bare Debian a lot. I know Proxmox is just software over Debian but you know what I mean. I am starting to have all my servers in plain Debian. Indeed moving to a homebrew NAS will mean moving from QNap's propietary thing to Proxmox with Debian VMs.

I have, anyway exceptions. Since Ghost recommends Ubuntu is is using it underneath.

General tools

So you can get a good view of the mess I am in I keep a Homepage installation with links to a lot of stuff here. It looks a lot because it is a lot but I think in a couple of iterations I can get a super fun start page. At the moment the mess looks like this.

You don't have to be super smart to notice I am mixing hosted stuff with remote stuff. I need a couple of iterations on this. I could throw some HTML and that is it but for some reason I am using homepage for this.

Also I have some concerns on storing secrets for homepage. The way it is done is not kubernetes secrets friendly so I have to play a bit here.

Anyway since it is not exposed to the internets I can live with it for a while.

Development tools

ArgoCD

Since I have a k8s cluster I need to have something to deploy stuff there. I am starting to have so many things that I need to keep everything versioned in version control.

Forgejo

I am moving some repos to Forgejo and using GitHub just for a backup. I am using forgejo actions (which are mostly GitHub compatible by the way) since I don’t need super complex workflows for anything.

Harbor

A simple docker image registry.

Longhorn

For the important storage I've been using longhorn last months. It is a very cool way of having storage redundancy in my K8S nodes.

Also the volumes in longhorn are the ones I consider critical so backups are regularly made to a Backblaze b2 bucket.

Prometheus and Grafana

One of the cool things of using Kubernetes is that you can use a Grafana operator to host the Json of the dashboard of every service in their own repo so you can compose the final picture based on the dashboards you have stored in every application repository.

It is not like I have absolutely everything under control yet but it is a small work in progress.

Development station

I still keep my dotfiles alongside my neovim config so you can see here how I work. I use XMonad in my main computer but I cannot consider myself a power user.

I host this at my Forgejo but it is doubled down in Github so it is easier to share with anyone.

Media consumption

Plex

A good old Plex. I could move to an Open source solution but I am too lazy to relinquish on my lifetime license.

Audiobookshelf

My audiobook host. I don't use subscription services like Audible so I have this so I can synchronize the position in the book from my phone and the computer.

Miniflux

My trustworthy RSS reader. I use it for blogs but it doubles down as a notificator for new releases of software I use. If it is hosted on Github or the like you can subscribe to the releases feed to know when an update is posted.

Also I use its integration with telegram to have notifications on my mobile phone when something changes.

ArchiveBox

ArchiveBox is Pocket on steroids and self hosted. You send it URLs and he archives them. Simple as that. It generates PDF files for everything which are the view I often use.

General tooling

Plausible

I have a plausible instance for analytics for this blog. It is not something extremely important but it is better than selling my visitor's data to Google.

Ghost

This blog is hosted on Ghost. Since now I have a remote Proxmox I can get small VMs for cheap so this sounds like a better way to ensure I continue writing.

Klipper

To control my filament 3d printer I have a small Creality sonic pad. This acts as a klipper server that has a webcam connected so I can see what is going on in the printer.

That is mostly it. It is a lot and it is overkill but it is extremely fun.

Anyway times advance and things move on. Lately I am hosting evilmeow.com, a Mastodon instance and the whole idea of the Fediverse is extremely appealing to me.

Being trapped in x.com should not be a thing and as of today it is. You cannot have a realistic alternative to the amount of audience there is in the old Twitter.

Federation has its own problems. There are defederation dramas and really dubious instances out there. But at least there is a way to work to solve those problems. It is true that there is not enough people yet in the fediverse and a lot of people you want to follow are on x. Anyway for me the pros outweight the cons. I am no celebrity so 🤷.

So when I saw this toot

I finally will have an alternative. I must admit there is stuff like Write Freely but they lack some features I want to have. Ghost is simple enough but feature full enough so I can use it and it is open source so...

It took me one afternoon to migrate in a dirty way. Mobile is broken as hell today but I need another hour with the stylesheets to have it.

Obviously my situation is just mine. Your case may differ but I am happy to see that we have alternatives and I want to contribute to the Fediverse being a better place. We are starting to have a lot of places that we can federate

• My mastodon: https://evilmeow.com/@rafadc
• My bookwyrm: https://bookwyrm.social/user/rafadc

More alternatives will appear but the important thing about this is that you create a ecosystem of interoperable tools with the things you need. I am not too much into putting pictures of myself in the internet but if I wanted I could always use https://pixelfed.social/i/web/profile/513318589511660378 and put more food pictures.

Every person is different and everyone will use different kind of tools. It is important that they are interoperable and replaceable. x.com, instagram and the like do all they can to retain you and not be interoperable. The fact that you can migrate your mastodon account to another instance and preserve everything you had is a killer feat every social network should have.

I am looking forward to see what system I am using in another 8 years.

Sprint is a book where I have mixed feelings. To be honest it is not for me. I am happy anyway that it is being useful for a lot of people. Its reviews are amazing so I am fine saying it is a good book.

Anyway, unfortunately I find extremely surprising people are finding it so useful.

For some reason we need a book for someone to come and tell us: "For Christ sake, you need to focus your full attention in the most important thing until the problem becomes a matter of execution" or "structured approaches are better than just throwing people at a problem" or "if no one here can take a decision... what the hell are we doing here?".

It offers a decent taxonomy and framework for working in complex problems but I find it too shallow.

I find far more appealing Cynefin taxonomy of problems and an executive mindset. The rest is "just" organizational problems and we have better frameworks for those.

I understand the way the book creates a framework that can be used for consulting purposes. Also, I am pretty sure those consulting services are super useful. Based on the reviews on the book a lot of people need it and I am super super happy someone is solving those. Anyway, as a society, why are we there still?

Why are we still questioning:

• Focus on solving the most important things
• Focus is mandatory
• Have input from all needed parts
• Consider what is happening as part of a system. Use mental models to approach those systems
• All models are wrong. Some are useful.
• Prototype early, fast and cheap

If you think of it the book is a huge exercise of moving from the proverbial Complex to Complicated

Maybe this is one of those cases where I find the book could have been a long blogpost but looks like I am alone there still so it is good someone took the time to do that.

In the book there are awesome examples of teams finding the solutions to great problems with thinking out of the box... and whilst the framework it is proposed is reasonable and necessary I don't think it is the most important part on getting to those ideas.

This kind of processes "just" (again with quotes) removes obstacles but I think they fall short on reproducing how those great ideas are created and that is the part I am interested the most. There are some small tips here and there on that but I'd love to see far more of that. If we assume we have a healthy organization that knows where they are going but they face a complex challenge you are already doing most of the things here.

Also another critique is that the things that are hard to solve are left unsolved. I have the feeling that it is focusing in organizations that are just close to solving the problem but for internal stupidity they don't do it.

I must assume most organizations are a mess. Bummer.

Anyway, again it is not that there is no bad advice there at all. Please, follow most of the things in the book. Maybe I am looking for solutions to stuff that no one can offer in a book.

Evilmeow.com

10/2022 - Active

My small mastodon instance. I don’t like the approach twitter is taking althoug I still have to use it to keep in touch with some people. Anyway I let my geekiness roam free here.

I may eventually open it to the public but it is something that won’t happen anytime soon. I am reading a lot about content moderation and probably it is not my cup of tea at the moment. The risk outweights the benefit.

I don’t have the production code in k8s since I made a dead cheap version to keep everything up and running keeping recurring cost under 20$ (for production alone).

Then the backups and monitoring are roaming free at my home. I am monitoring everything using Grafana and Prometheus and I am relatively happy with that part.

Keywords: ActivityPub, Ruby, K8s, Grafana, Prometheus

Micropomo

02/2024 - Active

Github

I needed a small pomodoro timer and I did not like anyone that was already there so I built my own

Keywords: Golang, TUI

No longer active Experiments

Software crafters Madrid

12/2012 - 03/2023

During 10 years I helped with the organization of a meetup in Madrid about software development.

This was a very interesting experience. I learnt a ton and hopefully gave something back to all peers.

The library of joy.pm

02/2022 - 11/2023

This was a crazy experiment to track the books I am reading. It was awesome to build a small rust tool I could use often.

You can find the notes for this in this blog post.

Actually I don’t use this anymore because I use bookwyrm and Obsidian far more.

Strato or tele

Website

This was a pet site to detect if a picture features a Stratocaster or a Telecaster. It is documented in this same blog.

I am not proficient in ML at all so this was a starting point for me. This was a fun thing to build.

It should still be working but I am not actively maintaining it.

Keywords: Machine learning, Python, HuggingFace

When doing Domain Driven Design we give a lot of importance to the modelling of the problem space we are working in. We should always do it in OOP but for some reason DDD has managed to strike a nerve we weren’t able to hit before.

But let’s start from the beginning. What is Object Oriented Programming?

In OOO we have objects that combine both data and behaviours. And they talk to each other in order to achieve a particular objective.

You can think that if you are using an object oriented programming language you are already doing object oriented programming style. It is not the case. There are a lot of situations that can cause not to be the case. Today I will focus on one of them.

Allow me to use Ruby as a driver for the discussion. I will try to keep it simple and understandable. I will make up also a “web framework” to put the focus in the topic at hand. Also I will be ignoring all dependency injection. I will maybe add a more complete example of pushing functionality to models in another post.

Imagine we are building a Book catalogue system. Somewhere in our system we have the following clases.

class Book
  attr_accessor :name, :tags
end

class Tag
  attr_accessor :name
end

At some moment we are receiving a web request so we add a tag to a book. We can have something like this

class TaggingController
  def add_tag(params)
    book = Book.find(params[:book_id])
    book.tags.append(Tag.new(name: params[:name]))
    BookRepository.save(book)
  end
end

Sometimes our objects stop being both data and state that talk to other objects and become just data structures. The importance is only in the things they contain.

We can take a bit of the behaviour and give it back to the objects for example in something like this:

class Book
  attr_reader :name, :tags

  def apply(tag)
    self.tags.append!(tag)
  end
end

class Tag
  attr_reader :name
end

class TaggingController
  def add_tag(params)
    book = Book.find(params[:book_id])
    book.apply(Tag.new(name: params[:name]))
    BookRepository.save(book)
  end
end

This is a subtle change, I know. I am trying to do the most minimal example. But even if minimal we can see some interesting changes:

• We’ve put a name to the process of adding a tag to a book. We decided to call that “apply”. Naming things is an important thing. This name should be decided alongside domain experts and become part of your ubiquitous language.
• We have changed from attr_accessor to attr_reader. We just constrained how a tag is applied to a book. The objects expose all that information.
• This is a bit contrived example since I want to keep is short but we can discuss if we want a tagging service that is the one that actually does that process. Normally I’d only have it if I consciously decide I want to have a Transaction Script style

If you compare the first objects to the second ones we can see that we are having more information about the domain model of the application too. They are not just bags of data. They are going from just data structures to objects.

It is not strange to see the first, more procedural, code style. We can start there as a mean to explore concepts and find good names for the domain model but eventually, if we want to follow an object oriented style we should move into the second direction.

Anyway, is this that bad?

Well, it is not like you are going to die in coding hell for this stuff but I find some advantages to the non-anemic model. It encodes more domain knowledge. Probably this is the most important thing for me. It forces you to put names to more activities in the domain so you can build an ubiquitous language for your context. That is often more valuable than any technical benefit. It is an opportunity lost in this context.

It is harder to model things this way, anyway. But that is not necessarily a bad thing.

Normally when we code in this style we end up either putting a lot of code in controllers or in Transaction Scripts. While having code in controllers is clearly a bad practice having transaction scripts may not be that much bad as a crime. It is just that is somehow makes weird the choice of object oriented programming for the problem.

If you are going full object oriented, let’s do it.

Happy hacking!

Resources

• Martin Fowler’s bliki on Anemic Model
• Martin Fowler’s bliki on Service Layer
• Implementing Domain-Driven Design

I am taking this time a more structured approach in choosing my sources. The course is indeed a non-deep overview on purpose and it is extremely practical. So I want to complement it with a couple of books. I am trying anyway not to use the books people say are the best but books that take a different approach on the same subject.

Take linear regression for example. I can read a very Python heavy book like Hands-On Machine Learning with Scikit-learn, Keras and TensorFlow that has a hands on coding approach and I can complement it with a visual guide on statistics like The StatQuest illustrated guide to machine learning that approaches the same problem in a different way.

This causes the same content to be received in different ways. This is not exactly multimodal learning but it is probably better than repeating similar sources. I still have to mix audio, videos and social interactions in the mix. Videos are already in the mix so I am going through some audiobooks on statistics. I cannot find any meetups in the place I am living now so I will try some online communities.

This may sound a bit weird but I still get super happy every time I can sharpen the saw a little bit more.

Happy hacking!

During all my career I’ve been avoiding to be identified with any technology. Instead, I’d love to be identified by the fact that I use technology to solve problems.

This has a couple of implications that I’d love to enumerate

Programming languages will change with time

I prefer not to be called a “Ruby engineer” for example. If I do that, I am losing the opportunity to learn from different contexts. We tend to stand in our echo chamber of our environment. We lose the ability to see different approaches to the same kind of problems in order to better understand the tradeoffs.

This is why I tend to use multiple stacks in multiple pets that are not the same stacks I use in my day to day job.

You can go wide but also deep in some things

This does not mean you have to avoid going deep in some stuff. It makes sense to become an expert in your text editor. It is just that you should not despise the rest.

I’ve been using Emacs for a long time but last year moved to neovim. It forces you to make a mental change that is well worth. I don’t regret any second I spent tweaking my (emacs config)[https://github.com/rafadc/emacs.d/blob/master/settings.org] and it helped me a lot in creating a workflow I am happy with for vim.

There are some things that belong to a common knowledge that is well worth the effort

I am finishing another skim of Designing data intensive applications and its knowledge can be applied to an enormous set of situations. It will not perish when new databases arrive or will not fade out when a new feature appears in PosgreSQL. It can be helpful when you are in a MySQL environment or in a MongoDB project. That knowledge is extremely valuable.

We learn by connecting the dots

This is often overlooked. There is one helpful strategy on learning that consists in associative learning. You learn better by connecting similar concepts together. Knowledge is better anchored if you have many different places where you get the same concept from. For example, if you know how to do concurrency in Go it will be far more easy for you to grasp it in Javascript. But also the other way round it true. You will become better and the concurrency you already knew by knowing what other options are and what tradeoffs other people had.

Also, this way of accessing the same concepts from different angles trains our ability to check to the same problem from different perspectives.

So, my recommendation to all my fellow inhabitants of the tech landscape is to prevent yourself from becoming a Java engineer and instead become a problem solver that specializes in tech to solve problems.

Happy hacking!

What’s this?

A test post to verify the build works.

I’ve been there. We have more books that what we like to admit and we have more games in Steam than what we’d like to admit. I was hoarding content for three more lives as if I was to get them.

This small rant is my own way of admitting that time is limited. Acknowledging that time is limited, anyway, has some implications.

Choosing your next read is a very rewarding task. You get the dopamine shot of imagining yourself when you already know the topic of the book. It feels great. Anyway when the time comes and you need to actually invest the effort in doing the read… well then that makes the thing harder. Reading is a high reward high effort activity in contrast to shopping books which is a low effort, low reward activity.

At the end of the day we try to spend a lot of time in choosing a new read since we are going to spend a lot of time reading so it may be worth the effort, isn’t it?

Well… it is and it is not. I spend too much time in small bursts of choosing a next read instead of actually reading. I could just concentrate that effort in a single moment making my decission more conscious and removing the background noise of all the other potential choices I could have taken.

Hoard and just choose when you have to

I am applying the same principles I do with software decissions. I delegate the decission to choose my future reading when to the moment I have to. This has the same advantages than with software. The best moment to decide what to read is when you have to. Your interests may have changed by that time or you have more pressing endeavours to pursue that make another read more appropriate.

You should either read stuff you can apply in your daily life or change your daily life so you can apply stuff you read.

Invest the energy in making reading cheaper

I am, lately, making reading accessible at all times. I store just a list of potential reads. Something dirty in an obsidian note. I add and remove things there with no thinking upfront. I just embellish the list when I am actively working on it finding my next thing.

Also I am starting to favour audiobooks that I can listen to while cooking or jogging. Something I can have at all times for short bursts of time.

Listening to this Ali Abdaal podcast has inspired me to start increasing slowly the speed which audio is played. I am not a native english speaker so I started slow but I am increasing with time and it looks like there is something deep there. It makes sense anyway. Listening requires less brain cycles than talking.

Leave reads unfinished

This is a classical advice I won’t elaborate here but I just will repeat it instead. Leave books unfinished if you are getting nothing from them. Do not suffer for nothing. Use your time wisely.

Happy hacking!

I will put more effort into evilmeow.com and probably will be thinking in opening it into the public with the extra headroom I get from this.

Still, the idea of having public facing social services is scary to me. The moderation part will be a challenge and I only have good will as a weapon.

Since we are in another AI surge I decided to write some pets in relation to machine learning. I was experimenting a bit with image categorization so I wrote one small web app. It is a super basic neural net to categorize if a photo contains either a Telecaster or a Stratocaster. Something that the world was not asking for but was really needed.

I built a small sloppy React UI on top of it. Do not be hard. It is my first React project.

Anyway the focus was on the machine learning side of things. Some learnings:

• Data cleaning is a PITA and takes a shitload of time. I spend a lot of time categorizing images. I tried to do a script to download a couple thousand from Duck duck go but at the end of the day I had to do a lot of validation manually.
• Iterating can get you to decent error rates. My first versions hace 30% error rate and I stopped at about 6 and taking into account that I am just a beginner probably this can be done far far better.
• Having a tough metric to fight like the error rate is super motivating. You run your training and evaluate your model against the testing set. It is super motivating to see how far you can go. It is far more rewarding than the green red cycle of TDD.
• I have a vast amount of things to learn. I am changing from resnet to levit because the error is lower. Why? YOLO!!! XD

It was easier than what I expected though. I am super happy to see than in an afternoon I can hack something relatevely useful. Let’s continue the journey!

Have fun!

The super obvious comparison is cryptos and the blockchain. We all have this in our recent memory so the comparison is evident. Anyway there are many differences. Blockchain was a nice technology looking for a good application. Cryptos were cool but… to much incentive to do the wrong thing.

I prefer to talk about two examples of the past instead. 3d printing and the iPhone.

3d printing never lived up to the hype. We all were thinking of a world with cheap goods accessible to everyone instantly and it never materialized. I am not saying 3d printing is not cool. I have 2 printers myself but it never lived up to the hype. It is a niche market at the moment that is absurdly fun but it didn’t transform the lives of 100% of people.

iPhone on the other hand stuck. It was an immediate revolution and its expectations lived up to it up to the point where every telecom company had to create their own mobile phone. Today, we cannot live without our smartphone.

Where will LLM’s fall? God only knows to be honest.

Anyway IMHO there is also a super meta discussion about how society is behaving about these changes that is worth talking about.

Companies are running like headless chicken to implement everything on an LLM even if it does not make sense. The context is that of a recession and investment is going to AI. To AI we all go! No one cares about building great products. VC money is still more tempting.

We live in a society where a lot of people has a mechanism of talking. Everyone wants an audience when few people want to really convey a message. There are tons of press headlines along the lines of “AI says it is going to kill all humans”. The fight for attention is terrible where the fight for quality is not there.

The first applications of ChatGPT are spam based things. We have people writing 38 books in Amazon in a month and another submitting PRs to hundreds of open source projects with code that does not even work. We couldn’t destroy the culture of pursuing overnight success.

We have now an AI generating the text for an email that will be summarized by a different email at the other end and probably we didn’t need neither the first nor the second. We are living a new tragedy of the commons.

Ok. I’m fine with that. Maybe my problem is with the definition of what means for a software to rot. If I had to give a definition to what I mean for software to rot I’d say:

The value a given piece of software provides changes over time. Almost always, this change is for its value to decrease.

That is what I mean with “software rots”. There are many reasons for that. Let me elaborate a bit more.

Software around you changes

Of course the most obvious case is that the software you have around changes. Let’s imagine we are building a software that downloads all your Facebook posts into a single file. Maybe that is extremely valuable to you. If Facebook API changes your software provides no value until you adapt to that.

Ok, ok. Contrived example. I know your software does not connect to Facebook so you are immune to that. Aren’t you?

For example in a previous job we used Redis. A lot. There was this funny little vulnerability. A Redis exploit that was discovered. Until we updated Redis our whole platform was vulnerable (in case somebody found a way to access it, but you can never say 100% you are secure). At that moment our software was less valuable for some hours. It is not a Redis issue. All the supply chain you are using to provide you service is a potential point of failure. Each day a failure is discovered you are vulnerable to an attack there and your value decreases. But wasn’t that vulnerability there all the time? Yes it was. But the fact that now is public has decreased your value a lot.

Also, you may be hosted on a single cloud provider. You are reliying on them to provide your value.

People change

There is this old notion of perceived value. Once your software is in production it starts losing the shiny brand new thing label slowly. The more time it passes users will start seeing its features as the new statu quo and it will feel less and less valuable to them.

Also, you are not the one writing software. There are more people providing brand new software that does something similar to what you software does. In a world with only one spreadsheet software this is extremely valuable but as time passes and when you have 50 different pieces of software doing that you need to do more things to provide the same value. It may be doing it faster, doing it cheaper or better but staying with the same 0’s and 1’s that never change will not make your software be equally valuable over time.

Is this always so hard?

To be honest I think it sounds harder than it really is. It just means that we need to strive to improve, slowly but steadily. It is true that a piece of code will always be a piece of code but software is just a means to an end.

Also, not all software loses value at the same rate. I’ve seen services stay in production for years with no change at all. That is awesome. We should design products for that, but we are not always so lucky to hit the right spot.

Ok, let’s be honest. This is overkill. This is too much. A simple notion page is enough for this but… this is 10 times more fun. Architecturally speaking this is an exercise in gratuitous complexity.

The finished product is in library.joy.pm.

What is library.joy.pm?

I take notes when reading books. But I do it in a very disorganized way. Sometimes I just scribble the pages (when it is a physical book) and some others I take some notes in notion. I also have plenty of markdown files scattered in many places. And don’t forget some emacs org files. I think I have those too.

This is my small attempt to become a bit more organized. I want to automate the boring parts and let me just do the thinking.

Just markdown files

Markdown files are absolutely awesome to keep notes. It is a simple format, it looks relatively future proof and you can version it in git. Indeed this same blog is written mostly using markdown and it works just fine.

So let’s start there. Let’s just create a git repo and put markdown files there.

I could have stopped there. Github renders markdown just fine and notes look great. You have a web interface and plenty of accessibility. It is more than enough for some geeky book notes.

A better visualization

Ok, let’s take another step just for the sake of it. Let’s build a website so I can present all those markdown files in a decent way. I could have used Jekyll or Hugo but I wanted to try something new. I like pandoc a lot. Sombody that keeps a huge Graphviz with 10000 intercrossed lines between file formats in their homepage must be nice people.

Pandoc is a great tool to convert between formats. We can use it to convert markdown to html.

pandoc -f drive.md -o drive.html

That would convert the notes on Markdown to HTML. Simple and neat. Pandoc supports templates too so you can just

pandoc -f drive.md -o drive.html --template layout.html

And that will apply the layout.html template to the output of pandoc. Let’s take a look how how templates look

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="/style.css">
    <title>$title$</title>
</head>
<body>
    <div class="container">
        $body$
    </div>
</body>
</html>

You have some template variables in the template. The body is $body$ and it will be substituted by the output of the pandoc. Now we have a beautiful stylesheet so we can see something decent.

Let’s apply a bit of magic pixie dust in form of a Makefile and we have our first version of a notes website.

This could be enough.

It would be great if we had the cover shown

If you paid close attention to the template I shown before you’ll see there is a $title$ variable. Pandoc parses YAML front matter in the markdown files to apply those variables to the template. So we can:

---
title: Drive
---

This is a great book on motivation.

[...]

And pandoc will substitute the title variable with the value of the title.

Also, I wanted to show the cover of the book. So I could add just some more metadata there and use The Open Library images to be shown there. According to their documentation we can create a link with the following url:

https://covers.openlibrary.org/b/olid/$openlibrary_id$-L.jpg

I could then just add

---
title: Drive
openlibrary_id: OL15016965W
---

[...]

And in the template use that HTML.

But that is a bit of a pain in the ass. I need to open the browser, look for the book and find the code. That is a bit time consuming for me. Also, that will make me consume bandwidth of the open library so I’d prefer to be a good citizen and host the images myself.

This may be a good opportunity to use a bit of Rust to create a command line tool so I can ask it to generate the template file for me.

The Open Library API is extremely simple and it does not need authentication so I can use it to get the cover and some metadata like the author of the books. I will add those to the templates.

Also, I can use direnv to change the PATH automatically when entering the code folder so I have the latest debug version of the binary easily accessible.

It is just writing post_template adding a bit of Tui-rs to the mix and I am ready to go!

0:00

/0:54

1×

Metadata as your database

Ok, so there we are. We can take notes and create new book pages easily. Anyway I have one last annoyance. I have to manually keep the index page adding every new book there and keeping the order. I could generate the index page automatically but then I can sort the new books by a single criteria. Having a static page is a must for me so I have to check other options. Let’s go overkill again.

I am accepting the metadata in markdown files as a the source of truth but that does not mean it has to be the only one. Since I am using a tool already to download the images and generate templates I can use the same tool to maintain a secondary metadata database.

I can maintain a sqlite database and upload it to the static site. Then I can use a WASM version of sqlite to load that database straight on from the browser and use it to generate the index page.

xhr = new XMLHttpRequest();

xhr.open("GET", "/metadata/books.sqlite", true);
xhr.responseType = "arraybuffer";

xhr.onload = function(e) {
    let database_data = new Uint8Array(xhr.response);
    const db = new SQL.Database(database_data);

    let sqlstr = "SELECT title, slug, finished_at, last_updated_at FROM books;";

    const result = db.exec(sqlstr, {});

    result[0].values.forEach(addToTable);
};

And yes, I could just create the table straight away but this looks absurdly more fun. Also it opens a lot of possibilities in order to display the information in funnier ways in the future.

Then I am using DataTables to display the index so it is still readable as I add books. Also, I get for free the sorting and searching features.

Finishing touches

Just some minor details. For example, this is the first site I use Ackee to gather visit statistics. It is self hosted and cookie-less so metrics are tracked in a privacy sensitive way. Let’s see how it works. I love the fact that I don’t have to set cookies for a web like this but I’d love also to have statistics on the number of visitors. On paper this looks like a reasonable solution.

What now?

I will start migrating more notes there. At this moment I just have a small PoC to check that everything is working properly. I could download the images in parallel, I could have a way to perform a full sync, I can show more info in the pages, I can add tags… This will have to be iterated. But this is the good thing on writing your own software.

Also with use I will discover more bugs in it. For example now I can easily my notes on a book. I have to take a look to that.

Why weird software?

This software could have been a Notion page. And it would have worked just fine. From time to time it is nice to relax and do some overengineering just for fun. And this is what it is. An exercise in having fun with your tools and your workflow. I will try to keep using it because after writing this I am a lot more involved with it than with a boring (but more effective) Notion page.

Also writing this software allows me to take some takeaways:

• TUI in Rust is easier than what I thought
• Embedding SQL in a webpage with no backend is a nice tool when you want to give users a lot of freedom to explore data
• Automate all the things!
• The Open Library is a beautiful project and it has a lot of potential. I ditched GoodReads already.
• I have no eye for design
• I should read more

I had loads of fun writing this. I hope you had fun reading this too!

Happy hacking!

I will focus in Jira for the sake of the storytelling but this is applicable to every single project management software.

During my holidays I invested some time in a beautiful book by Shane Parrish called “The Great Mental Models vol.1” [1]. One of the lessons we see in the book is “The map is not the territory”.

What do we mean by “The map is not the territory”? {#what-do-we-mean-by-%E2%80%9Cthe-map-is-not-the-territory%E2%80%9D}

What does this have to do with Jira?

At the end of the day we use Jira as a model with the real project. What is there is not the real project but a projection of the real project. It is a simplification over reality so we can do some operations over it with more ease.

Sometimes we forget that we are working with a model and we apply some characteristics of the real project to the model or even worse, the other way round.

Forgetting that a model may not be complete and still look good

Unfortunately the project needs to look good.

Having more people working on the model and too little on the real project

This is as nuts as having more cartographers than seamen.

Sometimes we spend too many resources making the model look good whilst the real project is an absolute mess.

Having a tidy model may give the false impression that the project is being tidy. Since most external stakeholders just see the model they won’t enter the complex world of the real project they will build their impression based on the model. This is a simple, intentional or not, form of deception.

[1] The great mental models vol1

Have fun!!

This is a controversial topic. I understand that some people need estimations and I may not be an authority to rant about this but… this is my personal space :-).

A lot of times when dealing with software projects we say we are optimizing for predictability. We want to build projects in such a way that we know beforehand how much it is going to cost. That may be a noble outcome but I think optimizing for that is a rabbit hole that can lead us to many problems that we didn’t consider in the beginning.

We are trying to fundamentally go to a model where teams are predictable and we are doing so by providing teams with the adequate processes and tools so they can estimate and do their jobs better. This sounds reasonable. In my humble experience, it is not. Not only unreasonable, it is toxic.

There is more than one fundamental problem here.

Turning estimations into commitments or even deadlines

When you want to be predictable you are aiming to be able to say “hey, this will be finished by this date”. Often, we do this by asking the team when they think they will have something finished and immediately assign that a certain degree of truth. No matter how hard we try science goes against us. This is not a problem that only software development teams have.

In 2002 Daniel Kahneman won a Nobel prize for “his groundbreaking work in applying psychological insights to economic theory, particularly in the areas of judgment and decision-making under uncertainty.”. This sounds a lot like estimations to me. And his findings are totally against what we are doing in software developmentworld. You can see an example in [1]. Estimations should be more “outside in” driven by comparisons and previous art than “inside out” by having engineers being in a negotiation to get the smallest possible date.

I don’t know when we turned this Agile trend into having a team of engineers agreeing on having this done by a date but we did. The more software teams I talk about estimations the more I see business using them as if they were kind of an informed guess instead of just a random number.

Kahneman found something really groundbreaking. Often, the more we know about a given task the more biased we are to underestimate it. Indeed, most of the times people that estimate tasks by comparing to other similar things without knowing the details get better estimations than the experts in doing that task. Kanheman has a Nobel prize. We don’t. Kahneman 1 - Software Engineers 0. Probably we should read less Uncle Bob and more behavioral economics but that is the topic for another blog post.

Then, are estimations worthless? Hell, no!

We should estimate. We have clear the concept that estimations should be made. But I disagree in two important parts. Who should estimate and what is the reason for estimating.

If you managed to read all this until now and you have learn a bit from Kahneman’s story, having an outside view is not only important but paramount in guessing when things will be finished. In my experience, in most of the companies I’ve seen only developers take part of it and the rest of the people in the room (if there is any) taking the “you are the experts” kind of attitude.

Why do we estimate then? We want to know when a project will be delivered. But if you agree on the Nobel-prize-evidence-based premise (wink, wink) why are we just onboarding engineers on it? Indeed they are the worst people to do it if you believe in the premise before. Why are our estimation process based on letting the devs guess instead of doing other practices?

What is worse. What do we do when things go south? You can change three things: scope, date or resources. None of those can be changed by the people you are making responsible of the dates.

Again let me use another fallacy of appealing to authority and cite Ron Jeffries [2] “Management has the responsibility to set the overall goals, the resources, and the date. The Product Owner, or the Customer, has the responsibility for giving us the best combination of features so that we meet as many of the goals as possible by the date. Development has the responsibility to deliver a smooth sequence of fully integrated software versions, at least monthly, containing an always-growing collection of running, tested features, as specified by the Customer and by Management.”.

Let me repeat. If someone is guilty of not making deadlines that is management because reaching dates is his job. I don’t know when they lost it.

It is always fun to try to explain your job to your grandmother when you are a software developer but, ideally, the purpose of your job should be simple and the goal of software developers should be “deliver the best software you can”. That’s it. They shouldn’t juggle on “should I add a one month buffer here so my job is safe?”. The management layer’s goal is “deliver on date” and take actions on the items we mentioned before if it looks like we are not reaching the date. We have agile coaches that are supposed to help the team to introspect so they don’t forget being a bit better each day. When did we change this?

Toxic implications in being predictable

We all want to keep our jobs. No one wants to look bad in public. If we are forcing people into being better at estimating and praising people when they get it right a lot of teams will start being predictable. Probably not because people do this intentionally but we will create a Darwinian process where people doing the wrong thing will be empowered and said they are doing the right thing. And that is terribly bad.

• Everyone will be adverse to risk so they will pursue worse outcomes in lieu of being more predictable.
• No one will automate. Why should we? If you do the same task over and over again you are able to estimate better how long it will take. You are able to create cookie cutter sized jobs instead of creating a cookie cutter machine so that task is no longer a problem. Indeed, being able to estimate reliably is often a sign of lack of innovation.
• Underestimating is punished, overestimating is totally fine. This forces everyone into leaning towards underestimating which creates the weird situation where we say that we can have something done by a given date we both lie conscious and unconsciously at the same time.

And sometimes we ask ourselves: Why building software in some enterprises is so hard? Mostly because they asked for it.

What is the model I am advocating for here?

Let’s be constructive and propose alternatives.

Think of teams as a black box. We need to ensure they are doing the best they can, whatever that is. We should be on top of them so they deliver “their best job ever”. Quote: “A business leader’s job is to create great teams that do amazing work on time. That’s it. That’s the job of management.” [3] The management layer over the teams should be on top of the team (not only the individuals but the team) and use all the tools at his disposal to train them to perform the best they can. The team should forget about dates and do their best. The management should juggle with dates, scope and resources to reach deadlines.

The people management layer needs metrics and tools to help the team find areas where to improve. Velocities, estimations, KPIs and such things are tools the manager has at his disposal so everyone can decide what to experiment on and check the metrics again to validate if those experiments are worthy or not.

I can only see two reasons for a team to do estimations. First, to serve as a way of discussing what needs to be done and align what the end goal is in the heads of people and second so the team can calculate velocity. If you do the first one you don’t even need to annotate the results and if you do the second one turning things into a date should be enough to detect you are doing it for the wrong reason :D.

A team should be closer to a sports team in the sense of pursuing improvement

A team should be closer to a soccer team than to a family [3]. We should be creating teams pursuing excellence instead of pursuing just getting the estimated date right. Estimations and in particular velocity calculations is just one of many indicators a team may have to self assess better.

If teams innovate and try to constantly improve they may have a panel of metrics and indicators on themselves same as we have Grafana dashboards for our software. It is fine if the team decides that velocity is one of the metrics they want to have. And to calculate that you need estimations.

So yes, sometimes we should estimate and it needs to make sense to track those estimations but what is important is the continuous improvement and not the dates we get from there.

[1] Kahneman excerpt from thinking fast and slow: https://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/daniel-kahneman-beware-the-inside-view#

[2] Ron Jeffries. The nature of software development https://ronjeffries.com/xprog/articles/jatmakingthedate/

[3] Patty McCord. Powerful: Building a Culture of Freedom and Responsibility

Have fun!!

Thank god some companies are starting to notice that not every single person in the company should be pushing features forward but some people should definitely be thinking on how we improve our daily work. Often, for some reason, that task fell into the agile coaches but IMHO that is not enough. They can think about the human interactions and the personal aspects of the job but there is some technical stuff we need to think about.

When I see that comic strip I always think that in developer time that may even be more because every automation you have scales proportionally with the amount of people that benefit from the automation.

I've just finished "A Philosophy of Software Design" by John Ousterhout just two days ago. It is a book on software design. I've been reading a lot about the topic of complexity lately and this came as a recommendation from Lee Hambley. Good stuff to come.

Some things I loved

• We need to talk about design: We need to talk about the interfaces we show in our code. We need to open the discussion to that kind of stuff that for some reason we give for granted. We are not formally trained in design but we do it all day whatever we want it or not.
• His naming is really good: He describes some concepts that are intuitive but provides a really good metaphor. The shallow vs deep classes metaphor is extremely useful and I will use it from now on.
• Care about interfaces: He spends a lot of time carefully designing the interfaces between pieces of software.

The things I didn't like

Part of the statements he is doing are extremely dangerous IMHO. It is always good to challenge assumptions but to my experience some things are plain wrong.

• Comments and documentation are not the same thing: I think he is mislead by the fact that all the languages he mentioned implement them the same way. He is making no distinction between one and the other and that is an unforgivable sin. Early documentation makes sense. Early comments are wishful thinking and mines in the land field. Elixir is an example of this. Since both are plain text we use the comment syntax to avoid syntax parsing. Anyway both are conceptually different stuff that just by chance have the same implementation. During the book I found both concepts confused one with each other.
• Tests don't only encourage only features: In this point I strongly disagree and I think he has a ton of misconceptions here. First thing is that he only considers unit and integration tests. Second is that I dont know how he looks for proof that his api is usable. Third is that IMHO test are most valuable when refactoring or when they are hard to write.
• He falls a lot in strawman arguments: This is something that I find extremely common in technical books lately. Some people white sentences like: "in general people do X". That is often a smell. You should critizise practices done right. It doesn't make sense to challenge something done wrong because it is obviously… wrong.
• A big block of code with comments is rarely more readable than smaller methods: I like that he tried to challenge this but TBH the discussion leads nowhere and the example he put is IMHO a counterexample of the point he is making. At the moment we've accepted in the industry The rule of seven plus minus two what he is trying to achieve doesn't make sense. For example:

def bussiness_process
  mutate_elements
  publish_results
  notify_client
end

private

def mutate_elements
  my_collection.each do |element|
    element.mutate_somehow
  end
end

def publish_results
  Sevice.publish(my_collection)
end

def notify_client
  Notificator.send(client)
end

will always be simpler to understand than

def bussiness_process
  # Mutation of the processes
  my_collection.each do |element|
    element.mutate_somehow
  end

  # We need to send the data to the final storage
  Sevice.publish(my_collection)

  # The notification is needed to fullfill ISO-god only knows what
  Notificator.send(client)
end

It will be harder to understand by definition because we have several abstraction levels in the same context. I wrote an intentionally short method in both scenarios to try to be as beneficial with his point as possible. I don't believe humans can understand two intertwined things in the same environment. The advantage of the small methods scenario is that we have the tool of scoping.

If you check the first example the explanation of business concepts is up to the documentation (see my comment before). The method acts as an index you navigate if and inly if (and this is extremely important) you want to dig into the details. And in the second you have only one potential way to see things. Everything. Even if I am reading source code I want to be able to layer things to the level I need. The second option is by definition less structured.

The small methods version is harder to write because it forces you to decompose the code, understand every detail and take the reader by the hand to explain what the heck was going on when you wrote that. The one method scenario is just easier for the developer.

There may be a case where the second is better but couldn't think of a single one that is not a extremely weird case that shouldn't be considered at all a general principle.

• He believes in love at first sight: Do two designs and you will know which is better when you see it. He is constantly taking decisions to be right. In my experience we are wrong half of the times so we design for change. We follow our hunches and we do the best we can.

In sum

When I went through my notes before writing this post I wanted to be sure I was not bashing this book at all. I think it is worth every second spent reading and all arguments I may agree or not are well explained and carefully exposed. It is also a very much needed discussion. I may disagree in some things but that doesn't mean is a bad book at all. Indeed after reading it I want to dig deeper in several topics related to software design.

This has been extremely useful for me in combination with Emacs but we will do one step at a time and we will start with the command line to move to more complex integrations.

Your first graph

No more thinking. Just fire your text editor and add create a file

digraph G {
  my_start -> one_branch;
  my_start -> another_branch;
}

Then turn it into a graph with the command line

dot -Tpng the_file_i_just_created.dot -o graph1.png

That will take the dot file you just created and turn it into this simple, but effective graph.

And we have a starting point. Maybe this doesn't look too impressive but we can see some things about the syntax of the dot language. We can also tweak the command line a bit to change the output format to pdf, jpg or post script.

We are defining a series of nodes and the connections between them. On the example above we created a simple chain but we can make it as complex as we want.

digraph G {
  step1 -> step2 -> step3 -> step4;
  step2 -> maybe_error;
}

So as you can see in the first part Graphviz is smart enough to arrange the nodes appropriately once you specify the nodes and its relations. It will try to organize the nodes top down in the more reasonable fashion.

Styling your graph

As a quick graph sketching tool this is turning to be awesome (if you use Emacs you can go to the end of the post to see how you can get extremely quick feedback) but if we want to put the graph somewhere we need to improve a bit the styling.

To improve how our graphic looks we can dive into the attributes of elements. For example we can specify the shape of a node by using the shape attribute.

digraph G {
  node_1 [shape=box];
  node_2 [shape=circle];
  node_3 [shape=plaintext];
  node_4 [shape=polygon, sides=6];
  node_1 -> node_3;
}

As you can see we can specify attributes of a node putting them in square brackets after specifying the node. But this is not limited to nodes but relations as well.

digraph G {
  node_1;
  node_2;
  node_1 -> node_2 [style=dotted];
}

And obviously we can play with color:

digraph G {
  node_1 [color=blue];
  node_1 -> node_2 [color=red];
}

And if you check the docs you can achieve even more complex styles

digraph G {
  default_node -> styled_node;
  styled_node [style=filled,color=".7 .3 1.0"]
}

But I will just cover the basics in this post. ;)

Org-mode

I started using Graphviz just for this reason and I think this is really where it excels. After reading the lesson before probably you have enough knowledge to automate something similar for your favourite text editor.

Using Graphviz to draw images

For all you emacs users out there you can use org-mode to render your diagrams. If you add them to a source block and C-c C-c you will see the following

 #+BEGIN_SRC dot :file my_output_file.png :cmdline -Kdot -Tpng
 digraph G {
   my_start -> one_branch;
   my_start -> another_branch;
 }

#+END_SRC

Automatically displaying images

In my case the images were not being displayed until I did an org-display-inline-images. We can add a hook for that to be automatically added for us

(defun my/fix-inline-images ()
  (when org-inline-image-overlays
    (org-redisplay-inline-images)))

(add-hook 'org-babel-after-execute-hook 'my/fix-inline-images)

Yasnippet

I also found extremely useful to define a snippet to create a new graph.

# -*- mode: snippet -*-
# name: dot
# key: dot_
# --
#+BEGIN_SRC dot :file ${1:file} :cmdline -Kdot -Tpng
digraph ${2:name} {
  $0
}
#+END_SRC

So I can quickly play with graphics in Emacs.

Scripting

Another funny thing you can do is programatically generate these graphs.

I created a sample script to create a graph of your followers on Twitter that may not be very useful but it can illustrate what we can achieve generating this graphs programatically.

Conclusions

This has proven extremely useful for me to quickly sketch out ideas. It is great to have Graphviz to care for all the positioning of nodes and just let you think on the actual connections about ideas.

This also opens the door for a lot of new integrations on emacs to generate images. I will work a bot more on that.

There is a Spanish saying that says “La felicidad va por barrios” which is used like “every dog has its day” but close-to-literally translates into “happiness goes to some neighbours”. Me and some friends wanted to check if happiness in Twitter depends on your neighbourhood too. Yes, I know it is a weak justification for writing software but… ¯_(ツ)_/¯

This is my first python code so don’t be hard on me.

Gathering data

The first thing we did was writing a small python script to fetch data from Twitter.

The relevant lines are like the following

api = twitter.Api(consumer_key=environ.get("TWITTER_CONSUMER_API_KEY"),
                  consumer_secret=environ.get("TWITTER_CONSUMER_API_SECRET"),
                  access_token_key=environ.get("TWITTER_ACCESS_TOKEN"),
                  access_token_secret=environ.get("TWITTER_ACCESS_TOKEN_SECRET"))

for twit in api.GetStreamFilter(locations=flat_locations, languages=['en']):
    if twit['coordinates'] :
        writer.writerow([
            twit['place'].get('name') if twit.get('place') != None else "",
            twit["timestamp_ms"],
            twit["text"].replace('\n',''),
            twit['coordinates']['coordinates'][0],
            twit['coordinates']['coordinates'][1]
        ])
        logger.info(twit["text"])
        csvfile.flush()
    else:
        logger.info('Discarding twit without coordinates')

I focused on twits in English for reasons we will see later. We will be dumping this data into a CSV file. Even if it is not completely optimal we will be flushing on every twit write since I want to be able to download periodically the csv to play with it meanwhile I connect to the stream.

Twitter stream allows you to filter by location and that is what I did. I was studying Spain and Germany so I defined bounding boxes for them. The bounding boxes are far from perfect but that is enough for the moment. The relevant piece of code looks like:

def flatten_array(array):
    return [item for sublist in array for item in sublist]

LOCATIONS = {
    'spain': ['-7.07','36.61','5.14','42.66',
              '-9.36','41.82','-1.58','43.97'],
    'germany': ['6.74','51.80','14.52','54.76',
                '6.15','50.86','14.80','51.80',
                '6.15','49.48','12.34','50.86',
                '8.04','47.50','13.13','49.48'],
}

flat_locations = flatten_array(LOCATIONS)

If you want to calculate your own bounding boxes you can use this web. The twitter API needs the array to be flat. Since I wanted to keep my locations as readable as I could I just create a function to flat the array and passed that to the api call.

Plotting the data to a map

I was using Folium to draw a map of the data I got. I did that because it was super super simple and I wanted something super super simple :D :D :D

Once I got my first map I started just platting markers on the appropriate places.

import csv
import folium

map_osm = folium.Map(location=[47, 0], zoom_start=5)
with open('data/twits.csv', 'r') as csvfile:
    twit_reader = csv.reader(csvfile)
    for row in twit_reader:
        twit_text = row[2]
        folium.Marker([float(row[4]), float(row[3])], popup=twit_text).add_to(map_osm)

map_osm.save('./data/map.html')

So with this tiny magic spell we can show something already. For example:

We can see here that the most populated cities in Spain get the most twits as expected. Will they be happy or sad?

Sentiment analysis

Since I am not very good at writing classifiers I will just use scikit-learn as a quick solution to get our sentiment analysis in place. We can create a linear classifier really easily.

I broke the classifier into a training script and another one to apply the analysis since I will be changing my CSV often but I won’t want to retrain the classifier every time.

TEST_SIZE = 10000

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

logger.info('Grabbing data')
training_df = pd.read_csv('twit_sentiment_analysis/dataset/Sentiment Analysis Dataset.csv', error_bad_lines=False)
examples_size, _ = training_df.shape

TRAINING_SET_SIZE = examples_size - TEST_SIZE

# Create feature vectors
logger.info('Training predictor. Grab some drinks. This may be long.')
vectorizer = TfidfVectorizer(min_df=5,
                             max_df = 0.8,
                             sublinear_tf=True,
                             use_idf=True)
classifier = svm.LinearSVC()

t0 = time.time()
train_data = training_df.head(TRAINING_SET_SIZE)['SentimentText']
train_vectors = vectorizer.fit_transform(train_data)
train_labels = training_df.head(TRAINING_SET_SIZE)['Sentiment'].map({0: 'negative', 1: 'positive'})
classifier.fit(train_vectors, train_labels)
t1 = time.time()
logger.info("Training took %d seconds", t1-t0)

logger.info("Dumping vectorizer and classifier for future use")
joblib.dump(vectorizer, 'data/vectorizer.pkl')
joblib.dump(classifier, 'data/classifier.pkl')

logger.info("Benchmarking training")
test_data = training_df.tail(TEST_SIZE)['SentimentText']
test_labels = training_df.tail(TEST_SIZE)['Sentiment'].map({0: 'negative', 1: 'positive'})
test_vectors = vectorizer.transform(test_data)
logger.info(classification_report(test_labels, classifier.predict(test_vectors)))

To be honest I still need to tweak the vectorizer since I don’t fully get all the parameters. I used the corpus at Thinknook to train the classifier. At the end I got a 79% of prediction which is not good at all but it is enough for me to have some fun. I will try to improve this in a further blog post.

Once the classifier is trained we can use it with:

logger.info('Loading model')
classifier = joblib.load('data/classifier.pkl')
vectorizer = joblib.load('data/vectorizer.pkl')

logger.info('Starting prediction')
twits_rd = pd.read_csv('data/twits.csv', names=['city', 'user_id', 'text', 'lat', 'long'])

twits_data = twits_rd['text']
twits_vectors = vectorizer.transform(twits_data)
prediction = classifier.predict(twits_vectors)

twits_rd['sentiment'] = prediction
twits_rd.to_csv('data/twits_classified.csv', header=False, index=False)

So using pandas we can easily generate another file ‘data/twits_classified.csv’ with an extra column containing either positive or negative depending on the classification of the tweet.

Plotting the results

Now we have all our data in place. We just need to modify a little bit the code block that does the plot:

for row in twit_reader:
    twit_text = row[2]
    if row[5] == 'positive':
        color = 'green'
        icon = 'smile-o'
    else:
        color = 'red'
        icon = 'frown-o'
    folium.Marker([float(row[4]), float(row[3])], popup=twit_text, icon=folium.Icon(color=color, icon=icon, prefix='fa')).add_to(map_osm)

And there we are:

Ready to take a look around to see happy people everywhere.

Conclusions

With this obviously unfinished we can see some things

• People tweets more happy feelings than sad. That great news. Maybe because now it is summer! Probably just displaying in a map won’t give a clear idea on the proportion between happy and sad tweets in an area. I will try a heat map in a following post.
• People in Spain don’t tweet a lot in English :D :D :D but I will analyze that in a different visualization.
• Barcelona and Madrid are the places that send the most tweets in English. Germany’s distribution is much much equally distributed.
• After reading all the data I had to filter out a lot of automatic tweets. There were some bots that tweeted the weather, a lot of Endomodo tweets, some people just posting the location… Cleaning the data a bit was not hard though.
• I got no clue on both machine learning and data visualization. I need to work a lot more on this. :D
• I need to be a lot more tidy while exploring the data I retrieved. Next time I will try different visualizations. I am learning though :)
• This was really fun exercise anyway. You should try on your own!

Have fun!!

I keep my config files on GitHub in an org mode file.

Since I am in an org mode the snippets and indentation and all the stuff are from org mode major mode which is a pain in the ass. For that purpose: org-edit-special to the rescue!

If we have an org mode file like the following:

* Homebrew

I'll add homebrewe'd emacs packages to load path

#+BEGIN_SRC emacs-lisp
(let ((default-directory "/usr/local/share/emacs/site-lisp/"))
  (normal-top-level-add-subdirs-to-load-path))
#+END_SRC

You can place the cursor inside the BEGIN-END block and press C-c ' to invoke org-edit-special. Then you will open a new window where you can edit the code in the proper major mode with all your awesome customizations and snippets in place.

Happy coding!

I have used several alternatives for different situations:

Writing pseudocodish code

I’ve seen a lot of people writing notes like:

{% highlight pseudocode %} Get all the customers Find the oldest group Send a complicated email to them Find the youngest ones Send a twitter message {% endhighlight %}

IMHO taking that as notes is absurd. How does that differ from throwing out a method that reads

{% highlight ruby %} oldest_customers = Customers.oldest send_complicated_email oldest_customers youngest_customers = Customers.youngest send_twitter_message youngest_customers {% endhighlight %}

Use the code as your notes. You are forcing yourself to make yourself clear there. Then you just need to fill the gaps.

If you want to TDD you can then start with the tests that will prove if that code you are trying to write makes sense or not.

Using a personal_notes.org file

In my emacs I bound F7 to a personal_notes.org file in the root of the git repo I’m currently in. That way I can quickly jump to a sketchbook where I can plate bits of text that I want to keep for whatever reason.

Using artist mode

If for some reason I need to draw some quick figures I can use artist mode. It is not extremely elaborated but it makes the trick a lot of times.

You can find an example of what I am talking about in this screencast.

Using org-capture

Org capture makes extremely easy to set reminders of things you need to do in the future. With a key binding I can save a small text regarding what I need to do plus a link to the location where I saw the problem.

I keep different files for different purposes. One for features that would be awesome to add, other for the code that should be refactored, other for topics to raise in the retro and some smaller ones for ongoing projects.

This allows me to just concentrate in the task at hand and leave the rest for a better moment.

Just not writing it down

Ok, you can do this on paper but this is a friendly reminder that you don’t always need to write down everything. Just let some things go. Storing all your ideas may be too much. Keeping track of notes has a cost too that needs to be leveraged.

So, let’s go to work!

I was trying the command line and it looked good enough for me. If you have aws-cli tools installed it is enough a mere:

aws polly synthesize-speech \
    --output-format mp3 \
    --voice-id Joanna \
    --region eu-west-1
    --text "We are about to have a lot of fun!" \
    hello.mp3

to get an mp3 downloaded into your machine. With that, probably we can have just enough.

Let’s go to emacs. I just added to my config the following function:

(defun my/elfeed-send-to-tts ()
  "Send current article to a text to speech system"
  (interactive)
  (let*
    ((html-to-read (elfeed-deref (elfeed-entry-content elfeed-show-entry)))
     (text-to-read (replace-regexp-in-string "<.*?>" "" html-to-read))
     (temp-input-file (make-temp-file "elfeed-input-tts"))
     (temp-output-file (make-temp-file "elfeed-output-tts" nil ".mp3"))
     (polly-command (concat "aws polly synthesize-speech --region eu-west-1 --output-format mp3 --voice-id Joanna --text \"$(< " temp-input-file ")\" " temp-output-file )))
    (progn
      (write-region text-to-read nil temp-input-file)
      (shell-command polly-command)
      (shell-command (concat "open -g " temp-output-file)))))

Elfeed retrieves the content in HTML so the first thing I have to do is get the raw text

((html-to-read (elfeed-deref (elfeed-entry-content elfeed-show-entry)))
 (text-to-read (replace-regexp-in-string "<.*?>" "" html-to-read))

Then, in order to be easier to me to work with the command line I just created a temporary file to store the text and a temporary file that will hold the mp3 of the text

(temp-input-file (make-temp-file "elfeed-input-tts"))
(temp-output-file (make-temp-file "elfeed-output-tts" nil ".mp3"))

I still need to investigate if the output file is correctly disposed since I will be overwriting it with the shell.

The rest is pretty obvious. We just prepare the command:

(polly-command (concat "aws polly synthesize-speech --region eu-west-1 --output-format mp3 --voice-id Joanna --text \"$(< " temp-input-file ")\" " temp-output-file )))

then prepare the input and fire the command

(write-region text-to-read nil temp-input-file)
(shell-command polly-command)

I’m using a little bash trick

$(< input_file)

That will put the file contents in the command. Probably I will have to escape the output somehow but this is my first approach. Let’s add this to the TODO list and continue.

Finally to play it I will use another dirty trick. This will only work in macOS at the moment.

(shell-command (concat "open -g " temp-output-file))

This will open the default mp3 player in the background and read the text.

With this we have:

So at least we have something working. We have still some problems with escaping the input text. Also I found that the command line has a limitation of 1000 characters. We will deal with this problems in further posts.

So I will point the main page of my site to my Jekyll and that is it. It is pushed to gitlab where the CI publishes it. I finally used gitlab since looks like the integrated Ci will give me more freedom to try some nasty stuff.

(Anyway probably I will check the crossposting plugin so I will still post to medium)

Yes, it is mostly text but it is super useful.

I send the entries that I need more time to read to pocket. I do that because I can use pocket client in my mobile phone or in the desktop or wherever the hell I am. I can also use any spare moment in the bus or the airplane even without internet access.

Elfeed by default has nothing like a pocket integration but as you’ll probably know emacs lisp is always here to help.

One of the less known pocket features is the possibility to send links via email. We will make good use of it. Remember that you need to send the email from the same email address you have in pocket.

In order to send emails I am using Msmtp. My configuration for gmail is something like:

account default
host smtp.gmail.com
port 587
protocol smtp
auth on
from my.email@gmail.com
user my.email@gmail.com
password application.password
tls on
tls_nocertcheck

Remember to create an application password for this. Never use your main password ;)

Msmtp is a beautiful tool to send email. You can send an email from the command line just with:

echo "Hey!!!!" | msmtp recipient@mail.com

That said let’s create the binding in elfeed config. If we check the source for elfeed show we can see that we can retrieve the URL of the current entry with:

(elfeed-entry-link elfeed-show-entry)

Piece of cake. Let’s then let’s define an interactive function to send the email:

(defun elfeed-send-to-pocket ()
  "Send current article to pocket"
  (interactive)
  (let
    ((url (elfeed-entry-link elfeed-show-entry)))
    (shell-command (concat  "echo '\\n\\n" url "' | msmtp add@getpocket.com"))
    (message "Saved to pocket!")))

I always add the final message to remember what is happening and avoid having an impossible to understand message like “Shell command successful” in the editor.

Now in my use-package for elfeed I only need to add the following in the config section

:config
  (bind-keys :map elfeed-show-mode-map
                  ("x" . elfeed-send-to-pocket))

This way when in an article I can just press x and the article will be saved to pocket.

Happy emacsing!

Things to consider before starting

I’ve always said that we humans have the beautiful power of learning from other people’s experiences so let me tell you mine:

• You will always lose features. Nobody knows completely what the application does up to the smallest detail. Nobody knows all the corner cases. Nobody remembers all the small decisions taking meanwhile wrig the application.
• A rewrite always takes longer than expected. Different orders of magnitude.
• Seriously. A rewrite will seem to be endless.
• A rewrite is a marathon, not a sprint. It’s chemotherapy, not some pills.
• The result may be better than what we had before in terms of speed, usability and maintainability. The problem is that the result may not be better and the customers may still prefer the old version.
• The sum of the cost may be smaller than the cost of evolving but you are not giving new value to the customers during the rewrite.
• If you improve the original application you are making the rewrite harder and maybe the rewrite becomes more expensive that adapting the old product. You are moving the target meanwhile you rewrite.
• If switching technologies you need all the team to be trained again. All of it. Not only coders.

Why a big rewrite looks awesome?

• It puts a “fixed” date to the end of all your problems. It brings hope. Anyway careful about short (4–5 months) schedules. When you have a cancer you should not trust the shaman that says that it can be fixed imposing his hands.
• A rewrite has not to fight with hard truths because it is not in production by definition. You are making an imaginary product fight vs a real one.
• New and exciting technologies! Often the technical team is keen on the idea because it gives them the possibility of ditching the old system with all its defects and play with new toys.

Things to NEVER do

• Switching technologies without taking the team into consideration. The team is going to maintain the product long-term wise. Make them part of it.
• Counter-strike rewrites: A terrorist comes, drops the bomb and then runs away. Maybe he hides and defends it for a while but he will finally gets the fuck out before everything blows up. Do NOT leave this to something whose future is not at stake in the rewrite. Do NOT make an external team alone to do the rewrite.
• Forget about the parts that brought us money of the old system since we are too focused on its defects.

When you should make a rewrite

• When you are changing WHAT the application does. If the new app is remarkably different it is perfectly fine to rethink and rewrite. It may even be good idea freeing your hands from old restrictions.
• When you have some constraint that forces you to rewrite. For example, the programming language you are using will no longer be supported or works only in a weird kind of hardware that you will no longer have access to.
• You have reached the technology limit. You are Facebook, or Google or Google and Facebook together.
• When you are fully committed to it. You need to allocate a huge chunk of time just to do it. Be prepared to invest a ton of time and money on it.

Bibliography

I leave this for the end because those are far better posts than mine:

• Joel Spolsky on why you should never rewrite
• DHH on their success rewriting
• Dharmesh Shah on rewrites
• A chapter of the bike shed on rewrites

We are going to compile the latest version of emacs.

Just:

git clone git://git.savannah.gnu.org/emacs.git
./autogen.sh
./configure --with-ns
make bootstrap
make
make install

I’m compiling the OsX version. In linux probably you will not need the with-ns flag. After some minutes you own built Emacs.app will be in the nextstep folder. The last “make install” does not actually install anything but it just copies several things into the Emacs.app so it is ready to work. That step deceived me a lot because I lost some time trying to make the thing work due to me thing that would install something.

It will take a huge amount of time but that is life.

The last 25 version (at the moment of writing 24 is in homebrew) is blazingly fast so probably I will try to stick to this for some time.

There are some things that are a bit unintuitive. You can directly alias emacs and emacsclient commands. I had to create two shell scripts in my path like

#!/bin/sh
/Applications/Emacs.app/Contents/MacOS/Emacs “$@”

Now you can have at last a working emacsclient -nw. Happy compiling!

I’ll break this into two categories: solving bugs and adding features.

Solving bug commit messages

The brief

Fixing code

Pros: At least it is not destroying code Cons: What the hell is happening there? I can commit anything under that excuse

More specific brief

Fixing bug

Pros: At least it is not introducing bug. Cons: What the hell is happening there?

Making it work again

Pros: It is working… again Cons: Why wasn’t it working before?

Fix bug in module X

Pros: Module X has one less bug Cons: At least I have a bit more context. Too bad I could deduct that information from the diff.

Fix #23344

Pros: I have a ton more context of the reason behind the error. Unfortunately I don’t have it immediately. If I use a modern issue tracking system I can navigate to a discussion regarding this bug from my browser or editor. Cons: I have to go to my issue tracking system

Fix #23344 Removing n+1 problem

The problem was that these lines were causing a nasty n+1 problem because we have a lot of entries in our “users” table.

Pros: I immediately have all the context of why the change was made and what the guy that had written this piece of code thought he was doing. Cons: A bit verbose sometimes

Adding feature commit messages

test / testing / just testing

Pros: The code is probably tested Cons: Everything else

Renaming method

Pros: Probably the new name is better Cons: I saw it in the diff already. Why it is a better name?

Refactor

Pros: No behaviour was changed. Cons: Just kidding. Please, tell me why you needed to refactor this.

Applying observer pattern

Pros: You know at least one design pattern. Cons: Why is that a good idea in that scenario?

Added new stories notification

This feature was requested in #112233

We are adding two new methods in the API so we can integrate in the future easier in our mobile app. These methods are now used in the frontend only.

Pros: A lot. Cons: Verbose

Putting it all together

When I look into a commit message, often, I am desperately looking for the reason why the change was made. I don’t want to know what changed. There is a diffing tool for that. It does not hurt if it is briefly describe what is done in the commit message but the most important part is why the change was made. When I read a commit message it often means that I am investigating a problem in the code so I need to know your assumptions to see if you were wrong or the assumption is now not valid.

Have some empathy with a person investigating something wrong with your work, most probably, your future self.

With the diff I can already see what changes were made. The text just explains what the hell I was trying to do and more importantly why. This gives the investigator a great deal of information to know if I was wrong in case something needs to be fixed.

This is one of the reasons I am starting to avoid using the -m switch of git commit

git commit -m “I did something with the code”

If we commit like that we are encouraging small commit messages and it becomes too tempting not to explain a little bit to the next programmer.

That is, also, why these days I’m embracing the idea of interactive rebasing my branches before merging to the main branch. This gives me some introspection time to organize the commits appropriately and reword some commit messages to make the lives of future readers better.

Now probably a ton of people will come with better ideas than this for commit messages. New ideas are welcome.

Warning: All the things I am about to write are my opinions and of course they are highly opinionated.

We had a discussion because of a candidate that came here saying that he aspired to manage a team of people in 5 years. In my opinion that is not what I’d like to see. Why? That is not the kind of ambition I want to see. That is climbing the ladder.

The question

What are your goals? What do you want/like to do?

I want to know what drives a candidate. What he likes to do with his life. I want to ensure that I am able to provide and he is aligned with us.

Not so good answer

I want to follow the career plan. I want to manage 5 people in X years.

You being in a position of having people under your responsibility is not a goal. Is a consequence. Also I’m hiring somebody whose goal is not what he is going to do. His efforts will be focused on switching positions instead of getting better.

Better answer

I’d like to learn X,Y and Z. I’d like to do A, B and C. All those things are supposed to be done here in the position I’m getting in.

I am able to say if am am able to make you reach your goals. I am able to see if you will be happy here. I am able to see if the company will get profit for you achieving your goals.

This is the ambition I love. You want to become better. It won’t be you against your peers. It will be you against yourself.

For me, finding a person that wants to become a manager is a warning point. Really. The best managers I’ve ever met didn’t want to become managers. Let me write it in a bigger font.

The best managers I’ve ever met didn’t want to become managers. {#the-best-managers-i%E2%80%99ve-ever-met-didn%E2%80%99t-want-to-become-managers}

If you are responsible for a team the worst thing you can do is become different. The kind of management that always worked for me is the kind of role model you can identify with. As a “managee” you want to somehow become that person. You don’t want that person’s position. He wants the same kind of things you want. You are all in the same boat.

If a software developer joins the team I work in I want him to become a better developer. That is what I aspire too and we all can go in the same direction. We can help each other.

There is some funny thing about the wrong kind of ambition. I think that deep inside it is only that they want a better salary disguised in wanting a different position. There is nothing wrong with ambitioning a better salary. But why don’t people want a better salary by being more valuable instead of being in a more expensive position? Why don’t you start in the kind of work you want to do?

Maybe all this is because I am a software developer and I deeply love my profession. I can’t see myself doing a different thing. Maybe it is because of that that I like to be in a corporate culture where we all go together to the same goal.

This is the first Microsoft product that I buy that feels really beautiful. It is really nice to have it in your table.

The position of the hands is really comfortable although a bit odd because it makes your wrists to stay horizontal. It feels strange in the beginning but the position is super relaxed.

There is a small detail I also love and it is that the spacebar is broken into two. Probably this is a really small detail but if I press it with a thumb the other thumb is not affected.

The keyboard is not bluetooth but comes with a small USB 2.4Ghz adapter which may not be ideal but is nice enough.

I don’t like noisy keycaps and in this laptop they are really silent. The pulsation is a bit mushy but if you come from a Apple keyboard like me this will feel super fine.

I was able to customize it to my taste using Karabiner which is an awesome tool for OsX. Also it allowed me to do some small hacks.

Remap CapsLock to control

Of course I do this to every new computer I get. This saves a lot of pinky emacs pain.

Switch between Alt and Command

In the sculpt keyboard these two keys are reversed in relation to where they are in a regular mac keyboard.

Use CTRL+IJKL for cursor

This is really awesome. I’m really enjoying this. Now I no longer have to move my hands to the arrow keys for moving the cursor. I find this also saner than Vim’s hjkl. I did this because in my opinion the cursor keys in the sculpt are really uncomfortable to reach and now I did this change also in my laptop’s integrated keyboard.

In conclusion, I’m super happy with my acquisition. This is a really nice addon to my setup. I’m looking forward to get really used to it and next year I will aim for a Model 01.

The simplest scenario

Let’s start in our computer all alone. Tmux sessions are run by a daemon the same way as screen does.

Let’s create a new session where we will pair

tmux new-session -s pair_programming
Now if you open a new terminal window you can attach your console to the same session.
tmux attach-session -t pair_programming

Then you are sharing the session in the same computer. Easy, isn’t it?

0:00

/0:10

1×

Even this may be impressive it’s not very useful.

In the same network

Now let’s try something a bit better.

We just saw that we can share a tmux session between two terminal sessions. Then we have a simple way of sharing the session over the network. We can simply make other user to ssh into your computer and run tmux from there.

Wait… Am I saying that it is a good idea to share your account with your pair? No, don’t get me wrong. We can do a couple of things to prevent this:

First, it is a good idea to create a local account only for your pairs. For example i’ve created an account called pair in my computer. Also it is a good idea to create a tmux group as we will see later. Add both your user and pair user to that group.

Also instead of sharing the password for that account I’m using certificates to handle who has access. That way it is super easy to revoke the access once it is no longer necessary. There is one awesome gem called github-auth that simplifies a lot the process. I can make something like

sudo su - pair
gh-auth add --users=<my pair’s Github user>

And it will allow my pair to connect to my laptop without having to deal with passwords or stuff like that. Also the pair user has access to tmux so he can connect to my session and finally he can’t access to anything else in my computer.

Also I can revoke access as simply as doing

sudo su — pair
gh-auth remove --users=<my pair’s Github user>

How does this work? Well, this works because all our Github public keys are public. You can access yours through a url like:

https://github.com/<your username>.keys

Then this is using ssh’s authorized keys to allow a user to login as pair user without having to type his password.

Let’s test this. I’m login with pair user and let’s see if I can connect to one of my tmux sessions:

tmux ls
failed to connect to server: No such file or directory

What? Isn’t this enough? Well… no. Unfortunately we need to add an extra small step. When we start a tmux session we need to provide a socket so it is shared between different users. Let’s start our pairing session with that new parameter:

tmux -S /tmp/tmux_pair_socket new-session -s pair_session

And we have created a new session and also exposing a socket so the other user can connect. The bad thing with this approach is that the socket will have wrong permissions for sure. That’s when the tmux group comes handy. I’ve created a /var/tmux/ folder and gave it 770 permission and changed the group to tmux. This way we can hold the socket there and change the property of the socket to tmux group.

mkdir /var/tmux
sudo chmod 770 /var/tmux
sudo chgrp tmux /var/tmux

Then let’s create the socket there.

tmux -S /var/tmux/pair_socket new-session -s pair_session

And we will be able to see it from pair user.

$ tmux -S /var/tmux/pair_socket ls
pair_session: 1 windows (created Sat Jul 11 11:40:08 2015) [78x30] (attached)

And attach to it with

tmux -S /var/tmux/pair_socket attach -t pair_session

So. After all this config I only need to ask my partner to do:

ssh pair@<my IP>
tmux -S /var/tmux/pair_socket attach -t pair_session

And we will be sharing the terminal session with a super fast latency.

Easing things a little bit

Anyway still the user I’m pairing with needs to know the location of the socket. Let’s make the things a little bit easy for him.

We will make a small addition to /etc/sshd_config file so the pair does not need to remember anything.

Match User pair
  ForceCommand /usr/local/bin/tmux -S /var/tmux/pair_socket attach -t pair_session

Then if some pair tries to login into my computer and there is no pairing session prepared it will only output an error and if there is it will directly connect to it. Piece of cake!

Also to start a new pairing session I’ve added an alias to my .zshrc

alias pair_tmux=’tmux -S /var/tmux/pair_socket new-session -s pair_session’
With all this I only have to type in a terminal:
pair_tmux

and my pair just has to

ssh pair@<my ip>

And if the credentials are correct we just need to start coding!

Also I have the following alias to add permissions

alias pair_adduser='sudo -u pair -i gh-auth add — users '
alias pair_removeuser='sudo -u pair -i gh-auth remove — users '

This way I don’t need to switch user to give permission to somebody to the pairing environment.

Firewalls

Let’s say that you want to host a session and you are behind a firewall. If you don’t want to change your firewall rules and you have a public host you can as a last resort try to do a reverse tunnel.

If you don’t have your own host you can try using ngrok. It creates a reverse tunnel for you so your pair can connect to it. We can just do

ngrok tcp 22

And we will have a tunnel open that will be available to your pair through the URL that appears in the screen. For example in the following scenario

Our pair will have to connect using the following command

ssh pair@0.tcp.ngrok.io -p 54147

And that is it!

Just as a piece of advice avoid the homebrew version and download from their website. Ngrok client is no longer open sourced and homebrew will not be up to date.

Ngrok will also be free as long as only one client connects to your TCP socket. If you need more that one person their pricing is really reasonable.

Summary

It takes a little bit of work to set up a nice setup to pair using tmux but it is worth the effort. In the works case scenario I only need to

pair_tmux
ngrok tcp 22

And share a URL for may pair to connect. Piece of cake. With this and a hangout I can pair with only a tiny little bit of lag and both users have control of the editor.