Dan Martensen

Software Engineering

Read this first

Exploring REST API Architecture

When building APIs to handle HTTP messages we’re faced with decisions about URL route structures, request methods, headers, payloads, status codes, safe and idempotent behavior, and more. Which if any of these are REST principles? In this post we’ll look at the initial REST architecture proposed by it’s inventor, Roy Fielding, including resources, representations, and examples.

REST Constraints

Fielding outlines REST as an architectural style for distributed applications with 6 constraints[1].

  • Client-server architecture - The most straightforward constraint, the client-server model makes clients, e.g. browsers and crawlers, separate from data storage. Clients make requests to servers and receive responses.

  • Caching - Responses are labeled as cacheable or non-cacheable and caching is restricted to the client or an intermediary between the server. This is referred as...

Continue reading →


Java Streams

Streams brought a functional and declarative style to Java. They convey elements from a source, such as a collection, through a pipeline of computational operations, But different than collections, streams don’t provide storage, return results without modifying the source, can be unbounded, and are consumable, where elements are visited once in each stream. Here we’ll cover how to create streams, common methods, and how sequencing operations affects output.

Example

Stream.of("acoustic guitar", "bass", "cello", "drums")
    .filter(s -> {
        System.out.println("filter: " + s);
        return true;
    })
    .forEach(s -> System.out.println("forEach: " + s));

Output

filter:  acoustic guitar
forEach: acoustic guitar
filter:  bass
forEach: bass
filter:  cello
forEach: cello
filter:  drums
forEach: drums

Each element moves vertically down the pipeline.

Stream pipelines

A...

Continue reading →


RabbitMQ Message Broker Patterns

RabbitMQ uses variety of asynchronous architectural patterns to decouple applications. Here we’ll cover round robin, next available worker, and publish/subscribe models, and features such as routing, pattern filtering, acknowledgement, and durability [1].

Message Broker

  • Producer - User application that sends messages to an exchange.
  • Exchange - Pushes messages into queues. Exchange types define what to do with the message when it’s received, whether. appended to a one queue, to many queues, or discarded. Example exchange types are direct, topic, headers and fanout.
  • Queue - Each exchange contains a buffer, or queue, that stores messages. A queue is only bound by the host’s memory & disk limits, it’s essentially a large message buffer. Many producers can send messages that go to one queue, and many consumers can try to receive data from one queue.
  • Consumer - User application that...

Continue reading →


SQL Performance of Join and Where Exists

Sometimes we need to identify record sets with at-least-one relationships. Find customers who’ve placed an order, products categorized as books, or cities that have airports. In this post we’ll compare the performance and execution paths of inner join to where exists in PostgreSQL 9.5.

Let’s settle with finding all products that have been ordered.

Setup

Create and populate two tables, products and orders, with 1,000,000 rows each of random data.

CREATE TABLE p
(
    p_id serial PRIMARY KEY,
    p_name character varying UNIQUE
);

INSERT INTO p (p_name)
    SELECT substr(gen_salt('md5'), 4)
    FROM generate_series(1, 1000000);

Output

|  p_id     |  p_name    |
--------------------------
|  1        |  5aHN4w0f  |
|  2        |  29LKwrBw  |
|  3        |  9cIR4iXE  |
|  4        |  5P9aTUQN  |
|  ...      |  ...       |
|  1000000  |  6cpGNL18  |

Pretty random looking. Now for...

Continue reading →


Rediscovering MVC and How to Write without a Framework

If you’ve paid much attention to front-end development in the last few years you’ve heard about Angular, Backbone, Ember, and other JavaScript MV* frameworks. They offer structure, bundled APIs and streamlined approaches to complex UIs, although not without concerns of performance, monolithic designs, and high churn rates[1][6].

All these frameworks share an adaptation of classic MVC, a pattern that transcends platforms, libraries, and languages, where models hold state and logic, views visualize state as output, and controllers handle input and interactions with the model and view[2]. Understanding classic MVC helps us evaluate strengths and shortcomings when selecting a front-end framework or micro-framework alternative. In this post we’ll cover MVC’s relevance, the roles of each component and how to write it with vanilla JavaScript.

Purpose

So why is MVC so prevalent? Some...

Continue reading →


Designing Normalized SQL Tables

Relational databases are reliable for complex queries, partial updates, transactions, and decoupling data modeling from application specific contexts, among other things. We can structure a relational database with a series of so-called normal forms in order to reduce data redundancy and improve data integrity. This includes organizing columns and tables to ensure their dependencies are upheld by database integrity constraints. It can apply to both new schema synthesis and current schema decomposition. In this post we’ll cover how to normalize tables in 1NF, 2NF, and 3NF.

NFL_63650.jpg

We have a table below of NFL data of players, teams, jersey numbers, stadium locations, and more.

Some issues off the cuff, right? What if I wanted to add a new team but had no player? What if a player plays multiple positions? Or two teams share the same home stadium? All these deal with data redundancy. And...

Continue reading →


Build Better Apps with ES6 Modules

ES6 is packed with features like iterators, generators, maps, sets, symbols, template strings and more. One of the biggest game changers to app architecture is the new modules, and different than the syntactic sugar of class and extends they’re an entirely new construct[1]. In this post I’ll cover ES6’s new module syntax and how it provides better performance, encapsulation, and error handling.

But first a little backstory. Engineers have long defined modules with workarounds on the global window object using object literals, the module pattern, and any combination of IIFEs one can imagine. Each of these comes with a set of challenges from lack of encapsulation to forced singletons. In 2010 RequireJS became a popular tool to define modules client-side, being used on sites like PayPal, Dropbox and The New York Times. CommonJS, it’s counterpart, was adopted server-side by the...

Continue reading →


Events, Concurrency and JavaScript

Modern web apps are inherently event-driven yet much of the browser internals for triggering, executing, and handling events can seem as black box. Browsers model asynchronous I/O thru events and callbacks, enabling users to press keys and click mouses while XHR requests and timers trigger in code. Understanding how events work is critical for crafting high performance JavaScript. In this post we’ll focus on the browser’s built-in Web APIs, callback queues, event loops and JavaScript’s run-time.

Code in action. A button and event handler.

<button id="doStuff">Do Stuff</button>

<script>
    document.getElementById('doStuff')
        .addEventListener('click', function() {
                console.log('Do Stuff');
            }
        );
</script>

Let’s trace a Do Stuff click event thru browser and describe the components along the way.

webapi5.png
From Philip Robert’s diagram

Browser

...

Continue reading →


JavaScript’s Map, Reduce, and Filter

As engineers we build and manipulate arrays holding numbers, strings, booleans and objects almost everyday. We use them to crunch numbers, collect objects, split strings, search, sort, and more. So what’s the preferred way to traverse arrays? For years it’s been the trusty for loop which ensures iterating over the elements in order by index.

In 2011, JavaScript introduced map, reduce, and filter as powerful alternatives when translating elements, finding cumulative values, or building subsets based on conditions. These methods help the developer manage less complexity, work without side effects, and often make code more readable.

In this post we’ll cover the usefulness of Array’s map, reduce, and filter methods. You’ll see use cases, code samples, behavior, and parameters of each method.

shell400-2.jpg
reduce() can find a Fibonacci sequence in O(n)

Drawbacks of Looping

Say I’m working on a...

Continue reading →


Converting a Tree to a List in JavaScript

With the meteoric rise of JavaScript MVC frameworks, data structures and algorithms long handled server-side are now managed on the client. JavaScript references objects thru memory addresses, making linked lists, trees, even graphs all possible to build and traverse. So dust off the old computer science textbook, grab a cup of coffee and whiteboard pen, and check out this interesting problem I encountered a couple months ago.

Scenario: The 3rd party web service your app consumes generates JSON in a tree structure with data values in the leaf nodes (example below). You need to convert these tree leaf nodes to a list for Angular’s ng-repeat to iterate over in your View. Not only must the nodes’ ordering be maintained, but any duplicate nodes should be removed.

funcobjbot9.png

Question: How can we convert a tree’s leaf nodes into a list while preserving order and removing the duplicates? We’ll...

Continue reading →