For instance today we are working on moving a domain from our internal DNS servers to AWS route53.

This domain has been in our hands for 25 years - we have a lot of DNS records.

We naturally have to make sure every record exists in the new DNS - otherwise people can't receive mails or other important stuff.

The natural thing to do is compare the two DNS servers. Unfortunately I don't know of a way to compare two dns servers.

And I promise you I am not going to do the boring work of comparing every DNS record I know of (I actually lied in the beginning - we are moving 3 domains).

But I am a human who knows programming.

So instead of sitting down and do dull comparison work - I will sit down and do interesting programming work.

Using the AWS cli for route53 I can get all the DNS records like so (if you don't know about jq and you do json stuff - you will love it!)

prod aws route53 list-resource-record-sets --hosted-zone-id $ZONE_ID | jq '.ResourceRecordSets[] | {Name: .Name, Type: .Type, Value: .ResourceRecords[].Value}'

I put this into a file and I do some manual mangling (I should automate that, but hey this is a one off - right [it never is]) I put it into a file called input.json

I then parse the file like to

jq -c '.[]' input.json | while read i; do
    echo "========================================="
    # Name, Type, Value
    name=`echo $i | jq '.Name' | tr -d "\"" `
    ttype=`echo $i | jq '.Type' | tr -d "\""`
    value=`echo $i | jq '.Value' | tr -d "\""`
    
    echo $name $ttype $value
    dig_options=( $ttype +noall +answer )

    existing=`dig "${dig_options[@]}" $name @8.8.8.8 | sed 's/[0-9]*//g' | sort -u`
    new=`dig "${dig_options[@]}" $name @$AWS_DNS_SERVER | sed 's/[0-9]*//g' | sort -u`
    if [[ "$existing" == "$new" ]]; then

      echo -e "\e[01;32mstrings are equal\e[0m"
    else
      echo -e "\e[01;31mstrings are not equal\e[0m"

      echo "from existing dns"
      echo $existing

      echo "from new dns"
      echo $new
    fi

done

This script will loop over all the records from route53 and look it up in dig from googles DNS server and the new AWS route53 server and it will give me a nice output, where red mean I made a mistake and green means OK.

Now all I have to do is run the script and look out for the red lines (and ignore NS and SOA records since the should be different)

What do I want to say with this? - well of course I am proud of what I did and want to brag about it.

But I also want you to think: When you have your next dull task with maybe a dash of repetitive work on top - "how can I automate this".

We know how to program - lets put it to good use!

As Mary Poppins would have said "Just a spoon full of sugar makes the medicine go down!"

The natural next question is then "How should I do error handling?".

That question can be hard to answer, and as always in software development the answer is: "It depends".

First of all error handling in every language is different, and you should certainly do what is idiomatic in your specific programming language.

Next it depends very much on you business case.

By making exception handling generic, you make error handling a technical thing - solved with technical methods and a technical mindset.

However when you do error handling you are, most likely, implementing something useful for the business you work for (if not you should probably stop and see why not). So we have to consider what the business needs us to do. If for instance you have a locked resource and your auth. system is down - do you give access to every one of do you shut out everyone? That is certainly not a technical question - somebody from the business side of things should be asked about that.

In the error handling, I mentioned in the last post, I btw. didn't find the error handling obvious: it took me a while to find out where the error handling was taking place.

The reason for this is, apart from my slowness, that the error handling did not stand out.

Error handling is the special case, and for a long time I tried to hide away the error handling, thinking that I did not want the special case (error handling) to stand in the way of the normal case.

However I have changed my mind on this.

For instance in go, error handling, done idiomatically, is every method that should be handled, returning a tuple of an error (if present) and the normal case return value. For every method call you have to check if the method succeed. When I first came to go - I hated it - but now I can see that at least error handling is in your face - you can see what is going on.

f, err := os.Open("/test.txt")
if err != nil {
    fmt.Println(err)
    return
}
fmt.Println(f.Name(), "opened successfully")

Bottom line:

Error handling is not a technical thing, but a business decision.

Error handling should be easily identifiable in the code, so you can see what is the normal case and what is the special case.

You might want to read the blog post I made about exception handling for junior developers.

The developers who made the api decided to be smart about error handling - instead of doing error handling in every api endpoint - why not just do some generic error handling.

The error handling exists in ControllerExceptionHandler.kt

The code would look something like:

    @ExceptionHandler(WebClientResponseException::class)
    fun handleWebClientResponseException(e: WebClientResponseException): ResponseEntity<Any> {
        logger().error("Error from WebClient - Status {}: {}", e.rawStatusCode, e.statusText, e)
        return ResponseEntity(mapOf("error" to "Error in external service"), HttpStatus.BAD_GATEWAY)
    }

That really seems like a smart thing to do - right!

Well I suppose you save a lot of time not doing exception handling on each and every endpoint.

However error handling is not something that should be generalized.

The thing is: when an error happens in one situation it should probably be handled completely differently from another situation.

For instance imagine a customer not being able to order a product on your website - you probably want customer support handing this ASAP.

On the other hand if you cannot load related products on the product page, you probably want to log an error and have developers look into it at next convenient time.

These are two examples that could easily both stem from the same WebClientResponseException - but they should be handled very differently.

Bottom line:

Exception handling is pretty important to do specifically and differently in every instance, since the response is probably different for different use cases.

And actually you most likely want to ask the business owners what should happen in each case.

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
Brian Kernighan from The Elements of Programming Style, 2nd edition, chapter 2

“Simple doesn’t mean stupid. Thinking that it does, does.” - Paul Krugman

Have you ever worked with a piece of that that made you think: "The person who created software was really smart - like a lot smarter than me"?

I have and when I was younger I was impressed, but as I have seen a lot of 'smart code' I am not impressed with it any longer.

Have a look at this Perl one-liner for instance:
find . -name '*~' -print0 | perl -0ne unlink

Honestly, I don't know what it does and figuring it out is almost hopeless. BTW Explain shell at your rescue

In the Perl community, there was a culture of making one-liner that would do magic. And that is exactly the problem with that culture - magic.

Code that is not simple and clear to understand is a pain to maintain and will slow you down. When the code is overly complicated and not simple, it could just as well be magic.
Remember: when we amend the code, we have to understand it first, and then we can change it. Of course, a good test suite will help you with the understanding and the confidence while changing it.
Some times I have seen people who did not try to understand the code before implementing a bug fix or a change.
This can of course be due to two things:

1. The code is too complex to understand
2. The developer is too lazy to spend time understanding before moving on.

The solution to 1 is actually to understand the code and then make time to make it understandable.

As a software developer, it is a pleasure to create a piece of code that is pretty complex - it makes you feel smart in - the moment.

But it is better to work a bit longer on this problem and make it readable and understandable for the next person, who takes over the code after you.

"One of the best programming skills you can have is knowing when to walk away for awhile." - Oscar Godson
"Sometimes it's better to leave something alone, to pause, and that's very true of programming." - Joyce Wheeler
"Best ergonomics advice is 'stay hydrated'" Michael Feathers

When working in a factory it is pretty easy to see if you are doing work if you are standing by the assembly line and do manual work of some sort - you are probably working.
Many people, even software developers, believe that writing code is what we do.
But I would state that thinking is what we do.

Most software developers have experienced working on a hard problem that they couldn't solve and then suddenly while they were doing some other mundane task, the solution to the problem just showed up.
So if you have a hard problem sitting staring at the computer screen for more than half an hour (give or take...) will not solve the problem.
Getting away from the computer will most likely help you.
I have often left work with an unsolvable problem only to show up the next day and fix it in 5 minutes.
Just to give an anecdote from the real life:
I once worked at a place where we had a strange error that we couldn't quite catch. We worked a long time to figure out what it was and couldn't figure it out.
But one day as I was sitting on the toilet with my phone browsing on the toilet it appeared to me.
I was not trying to solve the problem, I was just browsing the website...

And BTW drinking a lot of water will make sure you don't sit too long staring at the computer screen - especially as you get older.

"We value code that is easy to maintain over code that is easy to write" from Growing object-oriented software guided by tests.

"Quick becomes quicksand" Uncle Bob.
This is the first post in the Series on wise software catchphrases

This is probably one of the most overarching catchphrases in this series, I have often mentioned it to a junior developer while working on a task.

Just think about it for a second: You sit with the task of writing new code for maybe a day or two. After that, you and other people come back to the code again and again.

So often when we develop software we have a deadline or at least a product owner who waits for the task to finish. Completing the task quickly and cut some corners seems like a good way to satisfy everybody, and we can always fix the code later.
Thinking like this is what makes a mess of a code base. History and experience show that getting back to technical debt and fixing it later never (or only rarely) happens.
So if we follow the mantra of cutting corners to deliver quickly we end up with a codebase that has a lot of technical debt. If you do this over a period of time you will move slowly and the product owner will wonder why you are so slow, you used to be so quick.
Believe me, I have worked on code bases that had been developed in the spirit of delivering fast and in the end, it was not fast at all.

As you can see taking it slow will make it quick in the long run.
This particular catchphrase is, as I already mentioned, one that is overarching in this series, and hence the following posts will assume that we strive for code that is easy to maintain over code that is easy to write.

So I follow Code Wisdom on twitter:
https://twitter.com/CodeWisdom
What the account basically does is tweet quotations by famous and not so famous software developers. The quotes are wisdom gathered through many years of working with software development.

In my day to day job, I work with quite a few junior developers.
As a senior developer, it is my job to teach them the ways of software development; one of the things that I find most important for is to pass on some of the stuff I have learned.

But also I find it important to pass on some of the stuff other people have learned throughout the years.
Teaching them about my experience comes pretty naturally in the day to day tasks, but I thought it would be good to do a blog post series on some of the wisdom gathered by others along the way and add my reason why I find it valuable.
This way I can pass on the bigger picture of the software development landscape.

So in the coming weeks, I will publish a series of my take on software development wisdom. I will strive to make the post short and on the point.

1st post - We value code that is easy to maintain over code that is easy to write
2nd post - One of the best programming skills you can have is knowing when to walk away for awhile.
3rd post - Are you smart enough to debug your code?

So I attend a small church and I am responsible for the heating system.
That should be a simple task, but sometimes people fiddle with the heating system and sometimes the furnace simply stops.

And even though I live pretty close to the church I don't want to go there all the time to check the heat.

So I decided to setup remote heat monitoring in our church.

The way I have done it is I have setup a raspberry pi with a spark fun tmp102 temperature sensor and every 5 minutes, through a cron job, I send the data to AWS.

I decided to do it serverless since it is a way to small task to have a ec2 server running all the time.
I setup a lambda, where I can send a curl message to with the room and temperature.
I used the default template for a lambda and an API gateway to setup a Python lambda.

The following code is the lambda:

import boto3
import json


def respond(err, res=None):
    return {
        'statusCode': '400' if err else '200',
        'body': err.message if err else json.dumps(res),
        'headers': {
            'Content-Type': 'application/json',
        },
    }


def lambda_handler(event, context):
    operation = event['httpMethod']

    if operation != "POST":
        respond(ValueError('Unsupported method "{}"'.format(operation)))
    
    js = json.loads(event['body'])
    print(js["room"])
    print(js["temperature"])
    cw = boto3.client('cloudwatch')
    cw.put_metric_data(
    Namespace='temperature',
    MetricData=[
        {
            'MetricName': js["room"],
            'Value': js["temperature"]
        
        }"{}"'.
    ]
)

return respond(None, js)

This code is extermely naive and simply fails if not called correctly, for this use case that is just fine.
So if it is not called with room and temperature in a json it will return an internal server error. But I figure the only client is the bash script below and if it fails I will know through the fact that the metrics are missing.

As you can see this is really just the default template with me sending the room and temperature to CloudWatch.
I decided that if I wanted to go serverless Cloudwatch metrics would be the way to go.

So what is now left is to send the temperature from the tmp102 and send it to the lambda via a curl.

The following code is what I use to get the temperature from the tmp102 and send the curl

#!/bin/bash
hex_temp=$(i2cget -y 1 0x48)
dec_temp=$(printf "%d" $hex_temp)
echo $dec_temp
curl -X POST --data '{"temperature": '$dec_temp', "room": "kids"}' https://lambda-stuff.execute-api.us-east-1.amazonaws.com/prod/temperature

So the tmp102 sends the temperature in Celsius through hex. I convert it to decimal and send it in json to the lambda.

I set it up in a cron to send every 5th minute, and .... it didn't work.

After some googling I found this:
https://www.raspberrypi.org/forums/viewtopic.php?t=60702

What made it work was adding to the path of the crontab:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games

After that I simply called the above bash script every 5th minute.

And I now have a cloudwatch dashboard in AWS where I can see the temperature in the church.
And I can even set alarms if the temperature goes below or above a preset.

The other day I had a conversation with a Operations Engineer who made a bold statement: "Sending us a docker container is not different from sending us a .deb package"
I do not agree with that statement at all but didn't have the time to counter it, so instead I will write this blog post.

So what really is the difference between a deb package and a docker container?
Just a sidenote that .deb is just one example of a binary package, rpms and packages for other linux systems, could be another example. And I suppose the Windows term would be a msi package.

Well let me first quote our ops guy a bit more, he further said that if we need to monitor CPU usage for one application, we can just setup a vm for that application.

So the first thing to notice is that a docker container is not the same as a vm..
There are many differences between vms and docker containers, and I won't get into them all in this blogpost.
But one major difference is that a docker container is simply running on your computer (which is probably a vm by the way). And since it is running on your computer it does not have the overhead of a hypervisor and an operating system. These are pretty big overheads.
Furthermore a docker container does not really care which computer/vm it is running on, as long as it has a kernel that supports all the stuff it needs (such as cgroups, namespaces etc.)
This means that you are free to move your docker container around as you please and as you have resources available. This means you can use a container orchestration tool like AWS ECS or kubernetes, and hence use the resources on your servers more efficiently.
Hosting in a container orchestration also means that hosting a java or python app is basically the same, since your are 'just' shipping a container. Having big consistency on all your hosted applications means that hosting a new type of thing (be it language or external application) is not a big thing, since all it needs is a container, some configuration and some ports to expose.
It also means that the docker container is self containing (see a container is self containing) so all the configuration drift you get from long running vms should be gone.
Furthermore a docker container does not care if it is running on a developers computer, a test server or production - it just needs the correct configuration.
And since it is self containing it does not contaminate the computer on which it is running.
So a docker container will not have problems with applications wanting different dependencies (like java 7 and 8 or python 2 or 3), because everything is inside the container (<-- it is self containing).
If you start a docker container it will be fetched from the image repository and started. When you stop and delete the container and the image, it will be completely gone from the computer not leaving stuff behind that will mess up dependencies in the future.

There are many more differences, but these are just a few I choose to emphasize.

Exception handling is a very important thing for creating production ready code.
When you are a brand new developer you might not know how to do exception handling properly.
Here I will try to describe a few basic rules that should be followed when coding.
Please note that as always there might be reasons for breaking these rules, but they are a sane default.

The rules here can be used for java and c#.

Exceptions should be exceptional

Don't use exceptions for normal program flow.
In normal program flow use simple return statements.
why: Because exceptions were not made for normal program flow and hence they can be a very slow.
Furthermore the program flow is a lot harder to follow when exceptions are flowing around as a few methods might be skipped.
cure: Use normal program flow mechanisms for this.

Don't use Pokémon exception handling

You know the themesong from Pokémon: "Gotta catch them all" - well it may hold true when in the Pokémon universe, but when doing exception handling it rarely does.

Don't catch Exception.

why: If you catch Exception it is a strong communication that you do not know what you are doing which exceptions you are expecting.
And furthermore future exceptions that you do not know about will be handled
cure: Handle the specific exceptions that you know about and expect.

Don't swallow exceptions

Swallowing exceptions means catching an exception and doing nothing.
why: Only if you really know that this is by intention and always leave a comment as to why you swallow the exception. Not just a //do nothing by intention, but a real explanation.
Swallowing exceptions denies your future self the opportunity to debug the problem in production.
cure: Only handle exceptions you can actually handle

If you cannot do anything to an exception don't handle it

Let someone else handle it.
why: you are just adding extra code and taking away the chance to handle the exception where it actually has meaning
cure: Don't do anything where you cannot handle the exception, but do find another place where it makes sense. If you cannot find a place to handle the exception, maybe you have a problem in your code?

But maybe you should do some logging... But ...

Don't log an exception if you cannot handle it

why: If you log where you cannot handle it you will clutter the log and add no context.
cure: Log when you handle exceptions and add some context.

I have started a TIL.
To be quite honest, I don't post a lot on this blog, but on my TIL, I do post a lot more often.

During my job as a software developer I have stumbled upon some values that makes a lot of sense to me:

We value code that is easy to maintain over code that is easy to write
I am not sure if this is the origin of this value, but I first heard of this value in the book Growing Object Oriented Software, Guided by tests.
This should be the overarching value of 99% of all software developers.
You will have to go over your code again and again, and so will your co-workers or someone coming after you.
So the few minutes or hours you save now is just like wetting your pants, you will feel the warm now, but it gets a lot colder in time ;-)

We value code that is clear over code that is consise
This value is amongst other places found on the elixir-lang website
The simple reasoning behind this value is the principle that you read code a lot more than you write it, just as stated above.
One of the places where concise code has been valued too high is in perl one-liners.
Perl one-liners kind of made sense when using them on the cli. But they sometimes spilled into real code that had to be maintained.
For a period there was almost some kind of heroic thing in the software community, where the ones with the most concise unreadable code, that did the expected was supposedly the most heroic software developer.
I am glad to say that this is a dying culture.

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
This quote is from Brian Kernighan, and it probably one of the most clever things said in software development. I don't have much to add to it.

So as you can see these values are about almost the same thing: Namely writing code that is as easy to maintain as possible.
By writing easily maintainable code, you save the business many money in the long run.
The hard part about this is making the business understand that by not cowboy coding through this in a hurry now, you save them money in the long run.

So I decided to try something new:
I made a video explaining what an API is!

Of course if you are a software developer you probably already know, so move on then, or wait just a second - you could show this video to a business person you know :-)

So if you are, say maybe a business person, you have maybe heard developers talk about the API, and maybe you thought: "What the heck is an API?".
Well that is what I am going to explain to you in this video:

Now, what I explained in the video is that, basically an API is like a website for phone apps.
So you could say that you iPhone app browses a website, when it calls the API.
The app can both get and post data to the API, just like when you for instance search on google, then you post a search term like maybe: "new iPhone model" and google sends the results back to you.
Basically that is what the phone app does when it calls the api.

Just to make it clear, API means Application Programmable Interface. And it can be more than just what I am talking about here (which is an API for a phone app), but getting in to that would be too long haired for this post.

Working with Phoenix version 0.16 I needed to import some records from a csv file.
My first problem was where to place the tasks, after some research I found that you place them under lib/tasks and name them some_name.ex (not exs)

The code for the task should look something like:

You can now run the task with the command
mix task_name
That was the first problem.

Next I needed to use Ecto, in the task, since I wanted to import some records.

When I wanted to insert records I got the error (my app is called ExCMS):
repo ExCMS.Repo is not started, please ensure it is part of your supervision tree

It was pretty obvious from this error message that I should start the Ecto repo, and I found that I needed to run the following code in order to start the Ecto repo:
ExCMS.Repo.start_link

And all was good.

In Be My Eyes we use mongomapper and eventbus, both are very good tools that make my everyday life as a developer easier - except when it doesn't.
We had a problem where more than one helper is added to a call. This should not happen. We have a pretty simple if statement in our code. The thing is that many helpers were allowed through the if statement down to the "default" case, where only one should be allowed.
I really lost a lot of hair trying to figure out why more than one helper could fall through. It really made no sense, since we assigned a helper and assigned answered to be true. This should mean that the next time a helper tries to answer, she would be told that the call was already answered. But in many cases this did not happen.
I searched the codebase through and through for many hours with two things in mind that could be wrong:

1. The if statement did not work
2. The db returned something wrong.

No 1 is absurd so I did not follow that very far.
No 2 I could not make sense of until I did some research on stale objects and mongomapper.
I stumbled over and article on optimistic locking in mongodb and it hit me that somewhere in the codebase I must have and object with an old state that I somehow saved and by doing that, I overwrote the answered and helper attributes.
A quick Ag search found two places where I sent actual objects and not just Ids in the eventbus, after reading the aforementioned article I saw pretty clearly how bad this is.
For a quick fix I did a request.reload to get the new state before saving the new.
Long term it shall be a rule in the codebase that we do not send objects in the eventbus, but rather send ids. I already knew that and had it on my todo, but did not consider it important - well now I do ...