I really like listening to Software Engineering Radio episodes, especially when cleaning up the carnage once the kids are asleep. Over time I have gotten quite a lot of useful insights from the interviews. Kudos to the team!

279: Rust, borrow checking, third versions and verbosity

Link to episode

I forgot to take notes during this episodes so here is just what really stuck in my mind:

  • Rust’s borrow checker tells you for references, whether it is safe to use them with or without a lock. This is useful in a large project like Mozilla, because previously for safety reasons all references would be used with locks, to avoid conflicts, although this is often not necessary
  • Many Rust libraries are well-designed, because the people writing them wrote their third version of a library for this type of functionality
  • Rust often feels very verbose. This is intentional, because it makes explicit many of the things that remain unsaid, for example in C. The workaround is to create macros, that reduce the verbosity

273: How to get software estimation right

Link to episode

What I really like about this episode is how the interviewee first acknowledges, that estimates are a) often misused/misunderstood by Business as commitments and b) engineers often use bad methods (e.g. expert opinion) to make estimates, so that there is a reason they have a bad reputation.

His general opinion on estimates is that they should be like a suite case: you should get one that is big enough to handle the stuff, but not too big. It needs to be close enough, but need not be the perfect size. Maybe you will have to sit on the suitcase to get it closed, but that is OK. Main Takeaways:

  • Another way to put it, is that: Estimates are an assessment of capabilities to deliver something
  • Use historical data for estimates before projects start. On Day 0, compare the new one to previous projects and use as a base to say “it will probably take not longer than 2x this”
  • The best data for estimation is actual project data, so that should be used as soon as possible
  • Cone of uncertainty is what happens on a good Project
  • There is no sense in estimating in detail at beginning of a project
  • The intrinsic risks of a project usually dwarf the extrinsic risks by far, which is one of the reasons why estimates are so often wrong. The biggest intrinsic risk is that Requirements are misunderstood
  • Scrum is an estimation Nirvana - you estimate before the Start, keep estimating right after the beginning and changes that come in are also estimated. The biggest value is probably how accurate project data can be obtained very early, after just a few sprints (as opposed to months)
  • McConnell recommends a “Waterfall for requirements, development as Scrum” approach, to be able to give accurate estimates
  • One of the advantages of story points is that they scale with actual velocity - once velocity increases, you do not have to update all your estimates
  • Planning Poker is an instance of something called Structured Group decision activities - planning poker

263: What to consider before going distributed

Link to episode

I really like how Camille Fournier starts the episode with the classic definition of a Distributed System, that is even more true in the age of cloud providers: Your Machine does not work because some Computer you did Not know about failed.

Main takeaways from the episode:

  • Building a distributed system is now very easy thanks to cloud provisioning. Sometimes it seems as if we let Engineering build distributed Systems simply because it is fashionable and Fun. However the above caveat still holds: Building a DS always means introducing weird errors (Network, downstream errors…) you would not have without.
  • If you decide to build a DS think about which the Level of consistency every Level of Data requires. Then decide what software to use (e.g.Mongo for Data that can be reconstructed)
  • There are two real reasons for building a DS: Scalability and fault tolerance. Unfortunately the two work against each other

89: How Erlang grew and why it has immutability

Link to episode

Before listening to this episode I always thought Joe Armstrong would be some very academic and thoughtful guy who takes everything he says very serious, because Erlang is a language with such a strong “real engineers don’t mess around” attitude. After the episode I realized Joe Armstrong is a fun guy who developed Erlang mostly by playing around with some concepts and really enjoys his work and talking about it.

Some insights I got about the language:

  • Armstrong’s thesis is a nice write-up of the design rationales and decisions behind Erlang, along with some context. This is something I would welcome very much for most languages.
  • You could build an Actor-system without immutability, with isolated actors using message passing. The fact that Erlang has immutable data structures only is for fault tolerance: With immutability you are able copy the entire state of a process and run an identical copy on another machine (e.g. if two identical processes run in parallel on two machines and one crashes, it is extremely hard to restart the process on the crashed machine if you have mutable state, but trivial with immutable state).
  • The “Let it crash” philosophy means to discourage hiding errors, hidden by developers making ad hoc decisions how to handle unspecified behavior in low-level code. Erlang functions cover the happy path only, crashing if they encounter an unspecified input. In this case, the error is escalated to a higher level, where it can appropriately be handled (closing open files, resetting state) and making it visible by logging it. Only at that level are developers able to make informed decisions how to handle the unspecified behavior (e.g. checking back if this input should be inside the specification, escalating the issue, …).
  • If The Open Telecom Platform were called OTP on Rails, everyone would understand what it is.
  • OTP provides the usual application building blocks, but always with distribution and fault-tolerance in mind. The OTP DB Mnesia for example allows you to specify that you want a table RAM replicated on 2 machines and disc replicated one another one.
  • Erlang VMs carry the names of their creators: First the Joe Abstract Machine (JAM) and today the Bogdan Erlang Abstract Machine (BEAM).
  • Although presentations of Erlang usually start along the lines of “these are the requirements for a fault-tolerant, concurrent language”, this was not the way the language was designed or rather how it grew. Instead, Joe Armstrong experienced with different approaches in different languages to implementing a concurrency problem and Erlang grew out of his experiments with Prolog. Joe’s boss used to say “Erlang is whatever Joe defines it to be today”.

240: Newspeak and why static typing has disadvantages

Link to episode

I did not find the language particularly interesting, because it seemed really far from being usable for real applications. However it brings quite a few interesting ideas to the table:

  • In the language there is no global namespace, modules (implemented as simply as top-level classes) need to get passed in everything they require. This completely eliminates hidden dependencies and enforces dependency injection. Also creating a copy of the application process is trivially easy.

The author Gilad Bracha has some interesting comments on static vs. dynamics typing based on his year-long experience of developing both (e.g. the Java type system):

  • Static typing does not make your software more reliable. At runtime, the type system does not prevent errors.
  • Static typing benefits optimization is only very little - most JVM optimizations were originally developed for Smalltalk and Self, two dynamically typed languages
  • Static type checking can only show your program is not good (in the sense the type system defines it), but not that it is bad or produces errors
  • Static typing’s greatest benefit is additional documentation for humans and computers. By computers this means that IDEs can be more powerful using static type systems. However, many features commonly associated with statically typed languages (like refactoring), were not first developed for these languages (Smalltalk was the first IDE with refactoring support), which brings us to the next point, …
  • Type systems make it hard to invent things. In a statically typed language it is hard to be creative, to do things that the language designers did not think of. I felt this pain lately when I wrote some tests in Go with a colleague – the actual test and data were no problem, but most of our time and energy was spent on creating the type hierarchy necessary to satisfy the type checker