URL shortener in go

I recently combined all my various blogs into this one and then realised that, yet again, the length of my name is a pain. Especially when it comes to URLs. I also realised that I’d never tried to build a URL shortener, so I did. Full source available here.

There are a few tutorials around about writing one in PHP, but I decided to use Go. The basic idea is the same.

Database setup

The URL shortener uses a single table with the following structure:

Field Type Null Default Extra
id int(11) No None AUTO_INCREMENT
short varchar(8) No None
long varchar(64) No None
created timestamp No CURRENT_TIMESTAMP
accessed timestamp No 0000-00-00 00:00:00
access_count int(11) No 0

I think the fields are self-explanatory.

Code

The code itself is really simple. The idea is to handle every HTTP request and capture the path of the URI. Look the path up in the database, and return either a redirect or not found. I decided to use go-sql-driver for the mysql integration and gorilla mux for more advanced request routing.

func main() {
        flag.Parse()

        c, err := sql.Open("mysql", *user+":"+*pwd+"@("+*host+")/"+*db)
        if err != nil {
                log.Fatal("Failed to open sql dbConnection")
        }
        dbConn = c
        defer dbConn.Close()

        r := mux.NewRouter()
        r.HandleFunc("/{shorturl}", Handler).Methods("GET")
        http.Handle("/", r)

        log.Fatal(http.ListenAndServe(":"+*port, nil))
}

As you can see, all we do in main is open the connection to the database given the passed-in connection parameters, then set up a simple request routing table.

func Handler(w http.ResponseWriter, r *http.Request) {
        shorturl := mux.Vars(r)["shorturl"]                                                                                                               

        // lookup shorturl
        var longurl string
        var access_count int
        err := dbConn.QueryRow("select `long`,`access_count` from `url` where short=?", shorturl).Scan(&longurl, &access_count)

        switch {
        case err == sql.ErrNoRows:
                log.Printf("%q not found", shorturl)
                http.NotFound(w, r)

        case err != nil:
                log.Fatal(err)

        default:
                access_count++
                log.Printf("%q -> %q", shorturl, longurl)
                http.Redirect(w, r, longurl, http.StatusFound)
        }
}

The single handler is almost as straightforward. It queries the database to see if the short URL is registered and then either returns a redirect or not found, or fatals in the case of a SQL error. An extra feature that I might use later in a dashboard view is that it tracks the access count and last accessed time for each registered short URL.

We also don’t need to worry about little Bobby Tables as Go escapes the parameters for us.

Running

One final wrinkle is that I use Dreamhost for hosting and so can’t run a native Go server. However, with a little mod_rewrite trickery I can persuade Apache to forward requests to my server. Here’s the relevant lines of the .htaccess file:

RewriteEngine on 
RewriteCond %{SERVER_PORT} !^4242$ 
RewriteRule ^(.*) http://%{SERVER_NAME}:4242%{REQUEST_URI} [R=301,L]

This ensures that the given port isn’t already the one our server is listening on, and then redirects to the host with the write port number.

Next steps

Currently, short URLs have to be added manually through a SQL interface. Ideally, there’d be some UI for adding short URLs, and maybe even generating them through some hash function. Though checking for and handling hash collisions in a database is a pain.

All the small things

Someone, on checking out my github page, recently commented that I’m more of a breadth than a depth developer. I’m not sure that this is entirely accurate, though I didn’t take offence, but it is true that many of my personal projects are small one-off toys. This post is a tour of some of them.

dmanix

I didn’t study Computer Science at university1 and I sometimes feel like I missed out on some key programming projects like writing your own operating systems, compilers, and path tracers. I’ve been steadily remedying this and dmanix is my attempt at an operating system.

It only runs on 386 and has a scheduler, ramdisk, memory allocation, etc. The next step will be writing an ELF parser and compiling programs for it.

The only thing that separates it from other *nix OSes is that it’s written in more C++ than C.

lru-cache

One of the interview questions I ask on occasion2 is to implement an LRU cache in C or C++. The idea behind an LRU cache is that you have a fixed capacity cache in which you want to store data such that only the most recently used things are in it. In this case, I’m storing key/value pairs.

The specific reason why this is an interesting implementation is that it uses only operations that are average-case O(1). Ie, it should scale without significant performance reduction. This is managed by using both a list and an unordered_map; the unordered_map to store the key-value pairs for fast lookup, and the list to track how recently items are accessed.

buckets

There’s a well-known class of puzzles in which one is asked to fill a container using two other odd-sized containers. For example, fill a 5l container using a 3l and 4l container. I don’t enjoy these puzzles so I decided to write code to solve them for me.

It uses a depth-first graph search algorithm  to explore the solution space and then prints out the steps needed to solve the problem. There’s one issue which I should fix at some point: It might be better to use a breadth-first search as the solution space is relatively small and this would return the shortest solution.

queueserve

One of the things I’ve been doing recently is learning Go3. I felt that I needed to get to grips with the scalability and use for a web service, so I built a highly concurrent queue server and client.

It uses JSON over HTTP as a wire protocol and supports many, many concurrent clients thanks to Go’s built-in support for concurrent HTTP serving. The server is using lock-free queues to maximize concurrency.

evernote

Evernote have a series of coding challenges on their careers website. I enjoy a challenge so I tackled the problems. No-one ever contacted me so I don’t know if I did well, but my code passes all of the test cases and is relatively fast. All the other solutions posted around the web are in Python (that I found, anyway) so I thought having C++ solutions would be a neat change.

 

  1. Theoretical physics, if you were wondering
  2. and presumably can’t ask any more after posting this
  3. the language, not the game

gomud

I needed an excuse to learn go, and none of the usual candidate projects that I usually use to pick up a new language seemed to fit well. I was thinking back to how I started programming, and the transition from hobby programming to professional was largely thanks to working as a creator on Discworld MUD. It struck me that a MUD could be the perfect platform, given the support for concurrency and networking. And then I realised it would be even more interesting to write a MUD engine that others could use to develop MUDs. So I did: gomud.

Right now you can define rooms that have exits, players are persistent (though there are no passwords yet) and supported commands include ‘say’, ‘tell’, ‘me’, ‘shout’, ‘who’, ‘look’, and ‘go’.

Future plans include defining commands through data for easy expansion, add passwords for players, items, creatures, and then things like combat, skills, etc.

It turns out that it really is the perfect vehicle for learning go. The network support and concurrency in the language lend themselves really well to the core loop for a MUD, and the message passing works wonders for sending text to different connected players. Built in testing means that unit test coverage is strong, and it’s really easy to hack something up that works before iterating it into something more modular and well-structured.

If you can’t tell, I’m somewhat enamoured with both the project, and the language. I’m open to pull requests, so if you also want to get started with the language, or if you want to use it to build a MUD, please get in touch and help me build something neat.