2017 in retrospect - Markus Günther IT-Beratung

Well, we're finally here in 2018 - time to take a look at the past year. The year 2017 has been quite work-intensive, but also very rewarding in terms of my personal and professional life. This post will reflect on the professional aspects and summarize what I've been doing the whole year - and what my plans for 2018 are.

Research

Streaming Architectures

Most of my research time in 2017 was allocated for a deep-dive into streaming architectures. I worked on a couple of examples (cf. Open Source Projects) that showcase the design and implementation of applications that follow this architectural style. My interest for this kind of architecture and the technologies that enable it sparked as I got the chance to introduce Apache Kafka for a customer in 2016 to replace an existing legacy messaging solution that couldn't handle the growing demands in message throughput. I started off with digging deeper into Apache Kafka itself, talked about it then got into Kafka Streams after reading Streaming Architecture.

I published a couple of articles (in German) this year, ranging from introductions to Apache Kafka, Kafka Streams to data modelling with Apache Avro and building event-centric applications using Spring Kafka (cf. publications).

I also took a look at Apache Flink, which might be just your thing if you are doing a lot of complex ETL or have to be compliant with Beam dataflows.

Blockchain Technology

During summer 2017 I took some time to dig deeper into Bitcoin and its underlying blockchain technology. I started off by reading a couple of blog posts and I read through Mastering Bitcoin, which gives a great introduction into the inner workings of Bitcoin and blockchain technology. Even if you don't want to build your next blockchain app on the basis of Bitcoin, the read is quite interesting and entertaining and gives you the proper knowledge to get started. The author of Mastering Bitcoin is currently working on Mastering Ethereum, if Ethereum is more to your liking.

Fueled with my newfound blockchain knowledge, I started to experiment with a couple of use cases that - although minimalistic - had potential applications for small and large businesses from government agencies to enterprises.

However, due to the volatility of Bitcoin transaction fees, I wasn't able to come up with a pricing model suitable for a user-facing application, that is, a pricing model that is simple to understand from the perspective of a user and at the same time benefitial for the service provider. Although I discontinued my endeavors, the time working on this was well spent, as I do understand the technology better now, both in its (huge) potential and current limitations.

Open Source Projects

I open-sourced most of the code during my research on streaming architectures with Apache Kafka on GitHub. Other than that, I published my solutions for Advent of Code 2017, all writen in Kotlin. AoC was surely a great excuse for me to finally take a look at Kotlin.

Kafka-related

Other

Solutions to Advent of Code 2017 Puzzles implemented with Kotlin

Favorite blog posts

The world beyond batch: Streaming 101 - Tyler Akidau's article series on streaming applications is a must read for everyone that designs and implements those kinds of systems. Be sure to check out part two of the series as well.
Naivecoin: A tutorial for building a cryptocurrency - Lauri Hartikka explains what a cryptocurrency is made of - using clear and concise code. Highly recommended if you're into that kind of stuff.
A graph(ical) approach towards Bounded Contexts - Markus Harrer discusses some interesting use cases of jqAssistant for what can be best described as software archeology. The referenced article deals with the functional decomposition into bounded contexts using a graphical approach, enabled by the aforementioned tool. Be sure to check out his other posts on jqAssistant as well.
Building a CQRS / ES Framework - Renato Cavalcanti goes into the fundamentals of a CQRS/ES-enabled application and implements the necessary primitives to work with commands and events in Scala. He also did a thorough presentation on the topic during the university days at Devoxx 2015, which has been published on YouTube.
Hexagonal Grids - I stumbled across this absolute fantastic post from Amit Patel while doing some research on how to traverse hexagonal grids. The author presents various ways to establish coordinate systems on top of a hexagonal grid. A clearly written and thorough presentation of the topic, highly recommended.

Favorite technical books discovered and read

Designing Data-Intensive Applications - Up until now, there was no single book that I could recommend when it comes to fundamentals when designing distributed applications. Sure, there is Distributed Systems by Tanenbaum et al., which gives a proper introduction to the topic overall. Practitioners might long for a practical approach though and generally miss discussions of modern architectures and techniques. But look no further, Martin Kleppmann just wrote that book: Designing Data-Intensive Applications is a masterpiece and my personal book of the year. I cannot recommend it enough. This is a must read for any professional out there working on distributed systems, be it as a software engineer or software architect.
I Heart Logs - Jay Kreps, one of the founders of Confluent and author of Apache Kafka, discusses the log data structure in this short book. Due to its short amount of pages, it is a bit overpriced, but fortunately there is also a blog post by Jay that presents material similar to that in the book.
Cloud Native Java - Josh Long and Kenny Bastani provide a thorough introduction to Spring Boot, Spring Cloud and Cloud Foundry. The book also contains many best practices when it comes to designing various aspects of a Spring-powered application, which makes it even valuable if you do not commit to a fully cloud native application.
Migrating to Microservice Databases - This short book by Edson Yanaga talks about the challenges when moving a monolithic application backed by a relational data store to a microservice architecture. Along the way, Edson presents migration paths for relational data stores, talks about zero-downtime capability, CRUD vs. CQRS and discusses various integration strategies for your data. With 60 pages in total, this is indeed a very short read, but recommended nonetheless, as it focuses on migrating state when breaking apart the monolith. And this is a topic that is most often forgotten, resulting in distributed monoliths that are difficult to evolve. Highly recommended!
Reactive Microsystems - Another short book by Jonas Bonér that is essential reading if you want to understand the key ideas behind distributed systems that are both scalable and resilient.

Favorite Tools discovered and used

Lombok - I actually heard about it long before I used it. Makes working with Java less of a pain by removing boilerplate code (constructors, getters, setters, ...) using annotations. You should make considerate use of Lombok though, as some of its features may introduce subtle misbehaviors of your application - but that could be a topic for another article or talk.
TestContainers - Provides lightweight instances of infrastructure components, e.g. databases and the like, to JUnit tests for integration testing.
Insomnia REST Client - Insomnia is a beautiful REST client with an intuitive user interface.
JMH Visualizer - A tool for visually exploring the output of JMH.

Programming languages learned

Kotlin

Programming languages that I hope to explore more deeply

Kotlin - Advent of Code got me started with Kotlin and somewhat hooked. Looking forward to use Kotlin in other projects as well.
Idris - Very excited about this one, but haven't had the chance to properly look into it yet.

Conferences attended

Devoxx Belgium

Plans for 2018

Give one talk at a local JUG or similar local community gathering.
Write 2+ articles to be published in 2018/2019.
Write a couple of blog posts on current work- and non-work-related technology.
Read more non-technical books.
Research: Late 2017, I finally took some time to get acquainted with the Java Platform Module System (JPMS, aka Jigsaw). Although there are good books that introduce the JPMS available at present, none of these works goes into much detail on modularity patterns that are essential when designing modular applications. Since we - as Java developers - are stuck with the JPMS from now on, it seems like a good investment to do some research on modularity patterns and their application with the JPMS.
Research: Have a closer look at machine learning. Martin Krasser just published a list of reading materials suitable for an introduction to the topic.