This is part two of a two-part series of tutorials about regular expressions in Go. In part one we learned what regular expressions are, how to express them in Go, and the basics of using the Go regexp library to match text against regular expression patterns.
Regular expressions (AKA regex) are a formal language that defines a sequence of characters with some pattern. In the real world they can be used to solve a lot of problems with semi-structured text. You can extract the important bits and pieces from text with a lot of decorations or unrelated content. Go has a strong regex package in its standard library that lets you slice and dice text with regexes.
Text is all around us as software developers. Code is text, HTML is text, XNL/JSON/YAML/TOML is text, Markdown is text, CSV is text. All these text formats are designed to cater to both humans and machines. Humans should be able to read and edit textual formats with plain text editors.
This is part two out of two in a series on cleaning up data using Go. In part one, we covered the basic text facilities of Go and working with CSV files. In this tutorial, we’ll dive into actual data cleaning.
One of the most important aspects of any application is validating its input. The most basic approach is just failing if the input doesn’t satisfy the requirements. However, in many cases this is not enough. In many systems the data collection is separate from data analysis. It could be a survey or an old dataset.
This is part five out of five in a tutorial series on testing data-intensive code. In part four, I covered remote data stores, using shared test databases, using production data snapshots, and generating your own test data. In this tutorial, I’ll go over fuzz testing, testing your cache, testing data integrity, testing idempotency, and missing data.
This is part four out of five in a tutorial series on testing data-intensive code with Go. In part three, I covered testing against a local complex data layer that includes a relational DB and a Redis cache.
This is part three out of five in a tutorial series on testing data-intensive code with Go. In part two, I covered testing against a real in-memory data layer based on the popular SQLite. In this tutorial, I’ll go over testing against a local complex data layer that includes a relational DB and a Redis cache.
This is part two out of five in a tutorial series on testing data-intensive code. In part one, I covered the design of an abstract data layer that enables proper testing, how to handle errors in the data layer, how to mock data access code, and how to test against an abstract data layer. In this tutorial, I’ll go over testing against a real in-memory data layer based on the popular SQLite.
Many non-trivial systems are also data-intensive or data-driven. Testing the parts of the systems that are data-intensive is very different than testing code-intensive systems. First, there may be a lot of sophistication in the data layer itself, such as hybrid data stores, caching, backup, and redundancy.
Go is often used for writing distributed systems, advanced data stores, and microservices. Performance is key in these domains.
Go couldn’t load code dynamically prior to Go 1.8. I’m a big proponent of plugin-based systems, which in many cases require loading plugins dynamically. I even considered at some point writing a plugin package based on C integration.
Go has a very interesting type system. It eschews classes and inheritance in favor of interfaces and composition, but on the other hand it doesn’t have templates or generics. The way it handles collections is also unique.