What lessons has Advent of Code taught me this year?
Almantas Karpavi?ius
Software Engineering Team Lead @ OAG | Leading technical discussions, improving team processes, individually contributing
Practice makes perfect
"I do practice a lot at work" - you might say. That might be true, but unless your work gives you total freedom and you develop using the latest practices and technologies, you don't really practice at work. We do practice how to work with what there already is, but as professional software developers - we should also be able to innovate and that takes an immense amount of practice. The goal of this article - is to share how I practice and what coding kata using Advent of Code taught me this year. The topics I will go through are:
What is Advent of Code?
Advent of Code is an annual series of coding challenges announced at the start of December. Every day from December 1st up until December 25th - a new 2-part challenge is published. The challenges try to depict real-world problems illustrated through Christmas-themed stories. You solve a part - you get a start.
How did it go?
To me, the point is not collecting all the starts, but solving a problem in a way that anyone else could understand using clean code techniques that I know. Also, practice TDD.
I managed to solve 8 days (16 problems). The code metrics are:
As you can see, there is more code for tests than for problem solutions themselves. It took roughly 50 hours to get those 16 problems done with 100% code coverage. It took 15 hours more to write this article.
Link to my repo: https://github.com/Almantask/AdventOfCode2021
Lesson 1: Github Workflows and Repo setup
Every programming project should be code versioned. Code should be tested. Tests should run through a pipeline. I chose Github for versioning code, GitHub workflows for building code and running tests. I chose it because GitHub is the most popular git host and for open source projects it seems to offer the most.
Build and run tests
Previously, I would use Azure pipelines, but moving to workflows, surprisingly, was effortless. Here is the comparison of the pipelines that I have created of AdventOfCode for this and last year. Both pipelines build code, run tests and collect code coverage:
Integrating to static code analyzer - Codacy
Codacy is a free (for open source) static code analysis tool. It shows inconsistencies in your codebase, pinpoints potential bugs, bad practices, unused, inefficient, duplicate code. Everyone who takes programming seriously should use some static analysis tool. Setting it up was pretty straightforward - no YAML - just 4 simple steps (5min) linking your GitHub repo with your Codacy account. The benefit - grading my repository, exposing code smells, potential bugs and code duplications. All of it updated on git push.
Not only does it show issues, but also explain why is it an issue and often gives a recommendation, provides references as you can see in the picture below
Integrating to code coverage analyser - CodeCov
Code coverage is an important metric showing how many lines of code in % have been visited while the tests were running. Setting up code coverage took quite a few more steps than static code analysis.
First, Login to CodeCov using GitHub and link your repo in it.
Second, modify csproj (tests) file to include...
...coverlet.collector:
...and coverlet.msbuild:
coverlet.collector generates code coverage report (for example in XML) and coverlet.msbuild allows generating the report using msbuild actions (in GitHub Workflow).
Lastly, modify GitHub Workflow YAML...
...for tests to generate code coverage:
...and for the coverage report to be uploaded to CodeCov:
The result - with every push - your repository code coverage metrics are updated.
Badges!
I like badges - small indicators telling so much about the health of the codebase. Both Codacy and CodeCov support badges. You can copy-paste markdown from a dashboard of each and paste to your README.md as I did. I added a few more badges for the license, commits count, stars, contact information, communities. On the left side, you can see what they look like in my repository.
I used badges from:
You can have a badge for pretty much anything in any form you want. It simply is
[](Link-To-Information)
A good repository setup will allow others easier understand, use and/or join your codebase. However, repo setup is just the first step towards a smooth development experience. Moving on - to prep work for solving daily problems.
Lesson 2: Designing a good base
Before starting any project, you should think about the scope of it. Will you work on a thing for more than a few hours? Is it likely that different pieces of work will share similar traits?
Figuring this out is essential if you want to come out from writing code DRY. How do you write code, so that knowledge is not duplicated?
Abstracting Advent of Code day solution
This is the 3rd year I am doing Advent of Code. Every day is made from 2 parts. Every part takes a text file and outputs a single whole number up to 64 bits. To not repeat reading text, feeding the text to a component that can process it, printing out the solution, I have created the following abstractions.
IPartSolution - abstraction for solving a single part of a day
AdventOfCodeDay - reads an input file of a day, runs both parts of a solution and prints the results:
AdventOfCodeDay class is an abstract class, because it defines a structure and some implementation, but is incomplete. As you can see, I made full use of generics. TPart1 and TPart2 are both IPartSolution. They should both have all the logic they need within themselves. They are like mini-programs by themselves, therefore even if they have dependencies, it was okay to assemble what is needed inside them and not use Dependency Injection. No dependencies mean that they will have a parameterless constructor - that's why I could use a new constraint. Day is an abstract property and not a parameter in a constructor because this allowed me to simplify the initialization of classes that are generic. Generic constraint class is pretty optional - it is just for a good taste - enforcing implementation through a class and not a record or a struct.
The solution of every day follows can be seen on the left. Why did I choose to call the implementing class Solution instead AdventOfCodeDayX? I don't like verbose names. I trust the reader's ability to use fundamental knowledge and deduct the intended meaning. When a name is too long, you are forced to spend extra milliseconds reading code and that stacks up. In this case - namespace reveals it. To run all days solutions, I created a helper method to initialize a class object that runs both parts and did so for 8 days.
I find this code beautiful. This code tells a story. What does it do? It solves solutions of Day1-8. That is the beauty of code that speaks for itself!
To make this work, all I had to do is place input of all days under the same directory /days
Structure to decoupling what is asked from everything else
Every part solution can be divided further into 2 steps:
In other words, the nasty bits of parsing a text file, in which line is what word - has no significance in a problem that is being solved. For example, Day6:
You can spot 2 high-level parts: parsing input to build a component(s) that can solve the problem.
Extension methods for common operations
Text input in nearly every day shared the same traits:
领英推荐
For each scenario, I have created extension methods:
A strong base allows removing boilerplate and prep work needed for every further task. Next up - using tests to help design code and make sure it works.
Lesson 3: CTDD and the power of self-documenting tests
Test Driven Development (TDD) - is a methodology where you write a test first and then just enough code to make that test pass. This methodology helps to write minimalistic code and improves code coverage (because you are less likely to forget to write a test when you do it firsthand).
In this lesson, I will lay out my approach and the tooling used.
xUnit is the perfect fit for OOP lovers
Fact - xUnit is the newest test framework in .NET.
Theory - xUnit is the cleanest test framework when in hands of OOP lovers.
Why? xUnit uses intuitive terminology:
With xUnit, in most cases, you won't need any attributes other than the two mentioned (fact and theory), because OOP constructs are used in most steps:
FluentAssertions is the superior assertion library
Small things matter. I don't know about you, but I am not a fan of Assert.X assertion methods, because they are:
FluentAssertions library fixes all of the mentioned problems. How?
Self-documenting tests
Treat your tests as the code they test. Make them readable, reuse knowledge, keep them simple. Some say, that it is bad to hide things from a test, that it is okay to duplicate things. Once again, code duplication in itself is not bad as long as you don't duplicate knowledge.
That said, a test case should only contain lines of code that the scenario depends on. If a line of code is there just to make the wiring of a unit test happen - it should not be included in a test.
I could have written SubmarinePilotTests (day 2) like this - making the initialization of SubmarinePilot explicit. This makes code explicit, won't require you to scroll up and down to see the remaining context. However, do you really need to know how a pilot is created, to understand how it operates? Those are two different things.
I chose to eliminate the duplication and rewrite the test moving the initialization part to a constructor. Moving noise from the tests made it clear what important parts go into each test. Also, please note that I chose not to call _pilot.Move("up 1") and instead wrote a constant for it. I am a huge fan of 3A convention. The separation of a constant allowed me to be more explicit with what is an arrange (even if it is a constant) what is an act and what is an assert.
Here is an example (SubmarineControlsV2Tests) where I chose to duplicate the initialization of an object. It is good because I am testing the properties of the new control. If I were to reuse the one from the test setup - it wouldn't be very clear that it is new - it would require a reader to look for the existing context. Also, an act step (how I initialize the controls) would be hidden. The last example used AssertionScope. It allows executing all the assertions and aggregating all the failures, instead of failing on the first failed assertion.
When you look at the test name, it should be clear what you are testing under what preconditions and what the expectation is. Name like ShouldThrow, Ok, HappyPath, Expected - are not bad because in order to understand how a test works you will be forced to read its implementation.
NCrunch is the perfect tool for continuous testing
The best thing about tests - through one button click you can know whether the code you wrote works. What if I told you, that you can remove clicking any buttons altogether?
Continous test-driven development (CTDD) - is a way to run tests without any clicking, automatically, as we make changes. Visual Studio Enterprise edition has an option to do it, but from my experience, it never worked that well.
This year I have tried and bought NCrunch. It is easy to configure, is able to run just the tests that are related to the changes I am making and not only runs tests but also does code coverage and benchmarking. On the left, you can see green dots for passing and fast, yellow (the brighter it is - the slower the code) dots for passing but significantly slower lines of code. If code was untested, you would see a white dot, if a test was failing - a red one. Visualizing code like that makes it a lot easier to test untested code and find bottlenecks.
My experience of using NCrunch, as someone who essentially was writing a class library, was superb. The feedback loop was as short as milliseconds. There was no lag, no crashes. In my opinion - it is the best tool for everyone who practices TDD.
Why is it important to have 100% code coverage
With 100% code coverage I can make any kind of refactoring on a whim without fear of breaking anything. If I do break anything, I know immediately where to look and what to fix. 100% shows professionalism through trust - all this code is tested and works.
If it's 100% code coverage, it all must be working, right?
Code coverage isn't everything
The main problem with code coverage - just because you have visited a line of code, doesn't mean that it has been tested. That's why it's not enough to test all the lines of code - you need to go through logical scenario branches and test those.
For example, in ArrayExtensionTests we don't just compare 2 equal arrays. We also compare arrays of different sizes, initialized and not, empty. Even if it doesn't matter that much, it still gives us significantly more confidence that this code works. Code coverage does not show whether edge cases are tested. Don't forget to test them!
Also, with 100% code coverage, it is possible that tests are shallow. For example, there was no way I could test whether the startup of a program works in much detail. It might as well not be working correctly. But it was good enough for me to run this kind of test, a smoke test:
ProgamTests runs all the solutions and it checks whether the program does not crash. I used this test for Main because the problems I was solving did not have any output I could predict, so all I could test was whether I managed to implement the solutions using big input files without crashing anything.
This approach is useful, however, it is also unsafe. If I were to remove all the other tests - most of the lines would still be green, because this code visits every line of code in my source. Visited != tested.
Lesson 4: Retrospective to writing this article
Being a perfectionist
When I write about theory, I never leave it without practical examples that I have tried by myself. The theory is useless if you don't know how to apply it in real life. That said, sometimes I would write a sentence and then critically evaluate whether what I did was a good representation of it.
When writing this article, to solidify my points I spent hours on:
It takes time. But it helps me to both learn a few new things and solidify my knowledge.
Time management
As I was working on Advent of code, I had a lot going on in my life. I was (and still am!) editing a book. I am organizing Dungeons and Dragons (D&D 5e) game sessions for people at my workplace. I manage a community of C# programmers with over 7k people.?On top of that, I want to spend time with people that I love.
Activities such as programming are no longer my #1 focus. What I focus on now is the people that I love and I found that quite rewarding! My girlfriend has been nothing but supportive on this journey. We both sat down one evening, clarified our goals and made a draft plan of what, when and how. And the routine kind of stuck even after the Advent of code. I often reserve 1-2h a day for programming related activities. On weekends, I may reserve about 3h a day. Sometimes, I wake up 1h earlier and sometimes I do it after work. You take 1h here, 1h there and it stacks up to roughly 10h of practising a week.
I use the Pomodoro technique. Every 30 or so minutes, I take a break from mental activities and do a small chore, or check in with my girlfriend. It helps us a lot that she has her own goals and is on the same page about work ethics as I am. We both quit playing video games as we found better ways of using time.
I used to overtime after work. I felt like I wasn‘t doing enough even though I work really hard all day. I started taking small breaks and after work I do something else moving towards my personal goals. To my surprise, I am even more productive than I was before.
The point I am trying to make is that when you set clear goals and priorities, have a routine, you can achieve more than you think. However, you will always achieve more while getting enough regular sleep, taking breaks from constant focus and sharing support with others.
Pragmatism
I don't care about complete optimizations. I am okay with some duplicate code as long as it is not more than 10% of all that I wrote. I might have broken the principles I preach a few times in my code.
All that is okay. My code is not perfect. However, it was perfect to help me achieve what I wanted. What did I want? - I wanted to:
As programmers, we don't get paid for writing perfect code, we get paid for writing good enough code. A definition of perfect will be different based on the scope and importance of things. Do not chase some general ideal - define what you want to achieve and how - aim for that.
Conclusion
Repeating the things that you consider yourself to be good at will still make you even better. The key to growth is looking for ways how you can improve the existing ways of working. I dislike the saying "if it ain't broken - don't fix it". As a professional, you should at least consider whether improving something that already works gives significant enough practical value for a change to be implemented.
In this article, I have presented all that I have tried and learned during a month of Advent of Code. I want to grow as a professional - that's why I share what I have learned and practice even after work hours.
Practice makes perfect. Never stop growing.
Senior Software Engineer .NET
3 年Yippee. I actually finished it tonight. I learned a lot. The most interesting points for me: - xUnit is great for TDD lovers - Codacy is a static code analyzer (SCA) that works well with GitHub actions pipelines (or is this a library you install in the code?) - AssertionScope is a FluentAssertions (package) feature that ensures test execution does not stop after the first failed assert - NCrunch: A paid Visual Studio extension that automatically runs your tests as you write new code, giving you a tighter feedback loop about your tests and ensuring you don't have to remember to manually run your tests.
Senior Software Engineer .NET
3 年Yay. I'm halfway through and this read is super interesting so far. In particular, I find Codacy impressive. Also, I am inspired to set up a CI pipeline by myself using GitHub actions. We have both Azure DevOps and GitHub actions as options in my current team, but all the apps are currently built and deployed through Azure DevOps. I am seeing a great opportunity to practice with GitHub actions by moving CI for one of the apps from Azure DevOps over to GitHub actions. It's surprising how easy it is to read technical things when you're clear-headed. I realized that it's actually shorter than it looks because you added a lot of images inside the article ????. I'll be back.
Senior Software Engineer .NET
3 年That's a long one.