Containerization of Golang applications

I’ve been a working a lot in Golang recently and even though it easily allows for single static binary compilation I find myself using Docker a lot. Why? Well, especially when it comes to container orchestration and scaling of workloads some sort of container technology is used as these build the foundation of a Pod in Kubernetes (our deployment infrastructure).

While coming up with an initial Dockerfile is easy, correctly compiling a Go application and adhere to all sorts of container best practices is still hard. Does the size of the binary matter? If so, you might want to provide some additional flags during compilation. Do you build the app outside of the container and just copy the final artifact into it or do you use multi-stage builds to have everything within an isolated environment? How do you deal with caching of layers? How do you make sure that the final container is secure and only contains whatever is needed to execute your application?

The first thing I was overwhelmed with is how much old information is still out there and that even well-known developers such as Tim Hockin (co-creator of Kubernetes) are having questions on how to actually compile a Golang application correctly. As it turns out, some flags are strictly unnecessary as of Go 1.10 but are still widely used. Ultimately, it all depends on your needs and whether you need cgo or not but even after studying a lot of blog posts I’m still not 100% sure about my approach. As it turns out, Tim created a nice skeleton project which is a good starting point in my opinion.

Furthermore, I saw a lot of different approaches in terms of runtime base image and how the build process takes place. Some are using for golang:alpine with manually installing ca-certs, tzinfo, etc. during the build stage whereas others use plain golang instead. For the final stage common choices are either scratch or alpine which still provide a larger attack surface than i.e. gcr.io/distroless/base. As with many things, there’s not a single correct approach because one might want to keep the ability to docker exec -it into a container around whereas others have better ways to debug their services.

While coming up with my current solution I had the following considerations to take into account. Local development should still be fast, the build process must be CI-friendly with clean & reproduceable builds and no additional tooling needed to secure the final image such as microscanner or clair. Hence, I created a Makefile that helps me take care of the heavy lifting and allows for fast local development where no Docker is used at all. A shortened & simplified version looks as follows:

OUT := binary-name
PKG := github.com/package
VERSION := $(shell git describe --always --dirty)
PKG_LIST := $(shell go list ${PKG}/...)
GO_FILES := $(shell find . -name '*.go')

build:
	go build -i -v -o ${OUT} -ldflags="-X main.version=${VERSION}" ${PKG}

test:
	@go test -short ${PKG_LIST}

vet:
	@go vet ${PKG_LIST}

errorcheck:
	@errcheck ${PKG_LIST}

lint:
	@for file in ${GO_FILES} ;  do \
		golint $$file ; \
	done

container:
	@docker build --build-arg VERSION=${VERSION} -t registry/image:${VERSION} .

container-push:
	@docker push registry/image:${VERSION}

run: build
	./${OUT}

clean:
	-@rm ${OUT} ${OUT}-*

.PHONY: run build vet lint errorcheck

I’ll talk about -ldflags in a bit, so don’t worry about it for now. Since the regular go build command doesn’t do static analysis on the project files, I created steps like vet (checks for correctness/suspicious constructs), lint (style mistakes) and errorcheck (missing error handling) I can run whenever I feel like it. This is not done implicitly through another step such as build because my CI system takes care of these things too. The rest of the file should be self-explanatory if you’re familiar with make.
Now, the following Dockerfile is only used in my CI system for which I don’t mind it to fetch the dependencies during each build.

# Build stage
FROM golang:1.11.4 AS build-env

LABEL maintainer="Jonas-Taha El Sesiy <[email protected]>"

WORKDIR /project
ARG VERSION
COPY main.go go.mod go.sum ./
RUN bash -c "go get -d &> /dev/null" && \
    CGO_ENABLED=0 GOOS=linux go build -ldflags "-X main.version=${VERSION} -s -w" -a -o app .

# Final stage
FROM gcr.io/distroless/base
COPY --from=build-env /project/app .
CMD ["./app"]

I’m using multi-stage builds with the latest Golang version as the base image. For the final stage, I opted for distroless even though the final image is bigger than the other choices. Note that I’m using go modules for dependency management introduced in Go 1.11 for which I copy the go.mod and go.sum files into the container.
As mentioned before, there are a couple of flags passed onto the go compiler via -ldflags. -X main.version=abc allows me to pass on the version information to the binary which is then used within the app in some fashion. -s -w disables the symbol table and the generation of debug data in form of DWARF in order to reduce the size of the binary which is useful for my production image.

This is just my take on this. If you have suggestions for improvements or any other remarks, please reach out. Thanks! :wave:

Tags: docker, container, golang, go

DV lottery - What is that?, Part 3

Alright, this is the last part of the series on the ins and outs of the DV lottery. If you haven’t read the previous articles already, make sure to find out everything about the interview process in the first part and the some of the initial actions to take arriving in the country if you’re among the lucky ones in the second part.
In this part, I’ll be talking about getting a driver’s license, job search, and international travel as a resident (how to keep your status as a resident).

Driver’s license

As a newcomer to a foreign country you’re required to operate a vehicle at some point in time. If I think about my time as a tourist in the US it was easy as many states allow driving with a foreign driver’s license up to three months. Well, if you’re on an immigrant visa it’s different. In California where I live, the law says you’re only allowed to use a foreign license for 10 days if there’s an immigration intent. In crowded places like the San Fransisco Bay Area it’s almost impossible to get an appointment that fast.
If you’re hoping for a clear guidance on what to do in this case - I don’t have it.

Some people keep driving with their foreign license, some people use any of the available car sharing services to get around. The good thing is though, that once you passed the knowledge test you’re allowed to drive if someone at the age of 25 who owns a valid driver’s license accompanies you. This should help you to familarize yourself with your new environment and lets you practice for your driving test. For me the whole process took about 3 month and cost 35$.

Note: The US driver’s license is much more than just a license. It’s commonly used for age checks and serves as a valid travel document for domestic flights. Starting October 1st, 2020, boarding domestic flights requires the ownership of a REAL ID driver’s license or ID card. More info on this can be found here.

Finding a job

Even though becoming a US resident doesn’t mean you have to work for a US company or work at all but I assume that most of us need to make a living somehow so getting a job was one of my major concerns. Unfortunately, there’s no general rule of thumb I can provide you with because finding a job is strongly dependent on the industry you are operating in. For me applying through i.e. glassdoor wasn’t successful at all but instead going to Meetups and meeting people in person worked pretty well. I read that only a fraction of available jobs is being posted online and networking is so much more important than I was used to it from Europe. In general, I’d suggest at least 3 months in which one can settle and interview with a lot of companies to find a good job.

Travel

The article is getting quite long already but this is an important part, so bear with me. A big topic is travelling internationally as a resident as you’re supposed to be in the US. Nonetheless, having vacation and staying away for a couple of weeks is usually not an issue. If you can provide documentation that you’re actually living in the US and justify your trip, there’s nothing to worry about even with frequent travels. But if you plan on staying away longer than one year then you must apply for a reentry permit as your permanent resident card becomes technically invalid after this period.

A topic I haven’t covered here is the process of naturalization which means becoming a citizen. As I have yet to explore this myself, I can only point you to the official website.

This was the last part on the topic for now, please reach out if you have any questions or remarks!

Tags: diversity visa, green card, lottery, USA

DV lottery - What is that?, Part 2

As promised back in August, I’m finally continuing my post about the DV lottery. In this part I’m going to explain how I finally got the physical green card (compared to the temporary stamp in my passport) and what else I’ve been doing the whole time. Make sure to check out the first part if you haven’t read it yet.

After successfully going through the application process for the green card at the US embassy in Frankfurt, Germany I received my passport (with a temporary visa) and an envelope with my immigration documents. This envelope is sealed and under no circumstance are you allowed to open it or remove anything attached to it. The temporary visa has a shortened validity depending on the date of the medical examination plus 6 months. The soon-to-be resident must immigrate to the US prior to the expiration of this visa or the medical exam has to be repeated and a new visa needs to be issued. So here’s what you need to know about the pitfalls of using this visa until you receive the final document.

Immigration and Green card

The first thing to do is of course flying out to your prospective home country and handing over the sealed envelope at any port of entry. The officer will open the envelope to make sure the documents are complete and put the info into the system. He’ll then ask you about your home address because that’s the corresponding address for the United States Citizenship and Immigration Services (USCIS) to send your green card to once you paid the issuing fee of 165$. Furthermore, he’ll welcome you to the USA :+1:. What next? Well technically nothing needs to be done at this point in time except for wait for your social security card (which should arrive within 30 days) and the green card (arrival supposedly within 120 days) in your mailbox.

In my case this totally didn’t work at all so I needed to reach out to the Social Security Administration (SSA) and manually apply for the card. Also I started to getting appointment notices for the so-called biometrics appointment at the USCIS field office located next to my home address. These notices usually don’t apply for diversity visa recipients as they have their biometrics taken already at the embassys. Even after attending one of the appointments and having my biometrics taken again, I continued receiving these notices which proved my suspicion that something is wrong. I called the USCIS multiple times and was finally able to talk to an immigration officer who didn’t know what was going on with the notices and also not why there’s such a long delay in processing my case (120 days were long overdue at this point).
There was one last hope: The CIS ombudsman. The office of the ombudsman can be consulted for cases which are long overdue or in order to file complaints, etc. Two days after filing my case with them there was an update on my case (the green card has been mailed) and I closed my case with them immediately again which must be a coincidence, but usually this office is a good point of contact.
Just to let you know: Once the physical green card has been sent out, the temporary visa in your passport becomes invalid. I learned it the hard way through second screening in Austin, TX but that’s a different story…

SIM Card

As a German I was shocked about the prices for cellular service of any kind in the US. You can easily pay up to 70$ for unlimited data, text and voice here which is almost double the price of what I’ve been paying back in Germany but anyway… There are four major networks that operate in the US: AT&T, T-Mobile, Verizon and Sprint. I’m not going into detail on this any further except for this tiny remark: There’s a difference between GSM and CDMA, so make sure your phone is compatible ;)

Personally, I decided to go with what is called a Mobile Virtual Network Operator (MVNO). They share the frequencies of major networks (sometimes even all of them at once, i.e. Google Fi) and usually offer better deals compared to walking into the name brand stores. My MVNO is called Mint and operates on the T-Mobile network. At times, there might be reduced speed if the network is busy since direct T-Mobile customers are preferred over MVNOs but I haven’t experienced it yet and I think it’s fair with regard to the cost benefit. A downside of MVNOs is that they usually have very few or no stores at all, so you need to order it online which can be a little inconvenient especially if you need a phone right from the beginning (i.e. to Uber home from the airport). Be sure to check the coverage maps of the network operator, no matter if MVNO or direct brand as there might be huge differences depending on your area.

Bank Account

One of the first things you probably want to do is opening up a bank account. I’ve been a huge fan of online banks and looked into Simple only to find out that they’re exclusively serving US citizens. So I did some research for comparable offers and finally found Capital One to be a good alternative. They offer a great ATM network, no fees and everything is easily managed through their website or app. There are multiple types of bank accounts one can open but since I’m not here to provide financial advice by any means let’s stick with a traditional Saving’s acccount. Opening up the account is straight-forward online or through one of their very few branches across the country. All you need is an address for shipment of your card, a phone number in your name and your social security number.

If you apply for a credit card right away, you probably will be denied or get a very low limit on it. To build up credit, it’s best to start out with a Secured Credit Card. There’s another possibility if you happen have a credit card from an international operating institution such as American Express already in your home country. They offer to transfer your history with them and issue a new card for the US with comparable conditions as in your home country which simplifies things a lot.

Note: If you try opening up the account right after your arrival, there’s probably no credit history on you in any of the systems they check against. This means they won’t be able to verify that you are an acutal person and this is why you need to have a phone number in your name. I believe the idea is that i.e. if you went to an AT&T store to get a SIM card, they have already seen you in person.

Credit history

You’re new to the country and nobody knows about your spending habits and whether you are a reliable customer or not. Expect huge down payments and/or insane interest rates as companies try to mitigate the risk of you not paying your bills. A common approach to build up your credit without wasting huge amounts of money on interest payments is to buy everything with a credit card and pay it off in full every month prior to doing bigger investments such as getting a car or buying a house (if you can wait that long). Also make sure to put your name on the utility bills of the place where you are staying if possible which also increases your credit.

That’s it for now. I’ll continue in the next part on how to find a job, travel as a resident and much more, stay tuned!

Tags: diversity visa, green card, lottery, USA

Response to 'Why Bitcoin is failing the Muggles'

I recently read an article by Florian Gamper about how he thinks blockchain and crypto currencies - Bitcoin in particular - are failing the “muggles” - a nice reference to the average Joe taken from Harry Potter. Even though he made some valid points, I feel the need to make some clarifications.

In his rant, Florian starts with a short introduction and continues by expressing major concerns about 1. the environment, 2. trust, 3. security and 4. privacy. Each of these building blocks gives the reader something to think about and I’m going to stick to the scheme and comment on them sequentially. So here we go..

Introduction

In this part the author provides some good references for beginners and distance his writing from the current hype which led to failed ICOs and share increases through company renaming. Furthermore, he claims that “the community” describes the technology as magic which leads me to believe that he either went to the wrong meetups or talked to the wrong people.

I do understand that for the bigger part of the society it’s incomprehensable how the tech works and yes most of them can’t tell the difference between bitcoin and blockchain. However, that doesn’t hold true for people familiar with the matter and most certainly nobody of us would call it “magic” by any means. I must agree with his statement on the inflationary expectations though, as also Gartner confirms it.

The Environment

I really do care about the environment and I totally agree with all that has been said in this part. Unfortunately, the author only talked about one very well-known consensus algorithm instead of providing the reader with sufficient information on alternatives. He didn’t mention Proof of Stake, a secure and way more environment-friendly protocol (most notably implemented in Ethereum’s Casper Protocol) which would completely undermine his argument. Also, he didn’t take improvements to the Bitcoin network through second-layer protocols such as the Lightening Network into account which would allow vastly more transactions without the need for additional hash power. If you think that’s just a glimpse in the future, you’re mistaken as both solutions are either already deployed on corresponding test networks or even had a first live debut.

Another issue with this part is the made-up number of 80% of nodes being in China and the false claims about where the electricity is coming from. I haven’t found any statistics on which node uses which source of energy, however, I found the acutal node distribution as of Feb, 12th 2018:

Source: Bitnodes

There is also a whole different type of blockchain which is mostly being adopted in enterprises which use different consensus mechanisms such as Proof of Elapsed Time (PoET) or Redundant Byzantine Fault Tolerance (RBFT) as described in the Hyperledger Architecture Overview. They all consume tremendously less energy than the Proof of Work mechanism mentioned by the author.

Trust

This was really the part when I decided to clarify things. In my opinion, the line of reasoning in this part is.. let’s put gently: questionable. Blockchain enables trustless computing through it’s various components with the ultimate goal of reaching consensus. The author argues that the trust isn’t actually trust but faith as an algorithm determines it. Even though, technically speaking he’s right, he didn’t describe what the algorithm actually does in order to reach consensus which would give the reader a chance to judge for himself how much trust vs. faith there is. I’m not going to elaborate in much detail on how the transactions are being validated either, but an integral part of it is to check all transactions that led to the sender being capable of sending a transaction at a certain point in time. In other words: Before a transaction can be made, the funds necessary for this transaction are being validated. A technical description can be found in the protocol rules of the Bitcoin Wiki. From my perspective, this has nothing to do with faith.

The next argument was “money -> nodes -> owning the chain” and a presumptious threat if China’s 4 pools would come to an agreement. Well first of all, the referenced article about mining pool distribution is 7 months old which means it’s very much outdated. Just 2 months later, exchanges and pools started to shut down or move to foreign shores due to potential regulation. Second and more importantly, the author puts too much emphasis on the possibility of a 51% attack which even though theoretically possible and technically feasible, the damage caused is in fact comparably small as described here. From an economic perspective this attack can be considered infeasable with just a small gain compared to the effort put into it and an expected sudden decline of the token value if the network is compromised.

In the last paragraph the author refers to “other chains” and an even bigger threat for them to be exposed to the aforementioned attack. Since he’s not stating any specific chain, I can only guess that he means any of the recent super ICO coins. In fact, many of the coins are based on the Ethereum network and are so-called ERC-20 tokens, which would be indeed backed up by the underlying network for the exact reason of not wanting to create a new network, gain traction, build up nodes, etc.

Anyway, on to the next topic..

Security

In this part the author describes the pitfalls of public key cryptography and how it’s always been a challenge to safely store the private key in order to prove ownership. I do agree a lot with what’s been said and yes, storing the key in a safe place is a non-trivial task for non-technical people but for this exact reason there are plenty of wallet types out there to increase usability without compromising security. The latest trend is the rise of so-called hardware wallets, such as Trezor or Ledger Nano S. As stated, it’s not a problem with the technology itself and at some point people need to adjust to these kinds of things in order to reach mainstream adoption. From my perspective, this is mostly the same reason why PGP didn’t reach mainstream adoption even though it should.

Privacy

Florian again only focusses on Bitcoin in this section but generalizes it by saying blockchain. There are alternatives to prevent the mentioned traceability (also called linkability of transactions), such as Monero. Also, off-chain transactions haven’t been considered even though they provide enhanced privacy as they happen in private.

Wrapping it up

In my opinion, the author failed to be concise enough to provide some well researched information on the topic but instead leaves the reader - me - with a lot of questionmarks and unfinished thoughts.

If you have any remarks, please feel free to reach out! :wave:

Tags: blockchain, crypto currency, bitcoin, ethereum, consensus mechanisms, security, trust, privacy, environment