Struttin', croakin', moanin'!

Donut Tubing the prequel.

Robin Sloan makes a case for finished works, even while nurturing a feed:

Sometime I think that, even amidst all these ruptures & renovations, the biggest divide in media exists simply between those who finish things, & those who don’t. The divide exists also, therefore, between the platforms & institutions that support the finishing of things, & those that don’t...

Finishing only means: the work is whole, comprehensible, enjoyable. Its invitation is persistent; permanent. (Again, think of the Green Knight, waiting on the shelf for four hundred years.) Posterity is not guaranteed; it’s not even likely; but with a book, an album, a video game: at least you are TRYING...

Time has the last laugh, as your network performance is washed away by the same flood that produced it.

Finished work remains, stubbornly, because it has edges to defend itself, & a solid, graspable premise with which to recruit its cult.

The ol' Stock & Flow. Can't have one without the other.

In music, it's etudes and jams vs recitals and recordings.

Or perhaps you prefer keeping a sketchbook while working on paintings.

The secret I see working the best for folks is when they can gather up their flow and make stock out of it.

I can't help myself from watching episodes of The Computer Chronicles. These are just tremendous time capsules.

Here's a particularly fun one: Audible has been around longer than you might have though. Below is Stewart Cheifet introducing the service in 1999 (from the Y2K special!):

Amazing to see the sophistication of the player. Not too different from how the iPod handled automatically downloading and clearing your subscribed podcasts.

I was in need of a detour. I've been pretty focused on large projects in the web dev category. So, to scratch the curiosity itch, I hacked away at a very simple iOS app.

I had a few questions I was looking to answer going in. I was largely curious about how big the gulf really was between web development and native.

Additionally, what are the benefits of fully committing to an ecosystem? VS Code is fine-tuned for JavaScript and web development, while Visual Studio is a full IDE for C# and .NET. In that sense, I had some experience, but I was still curious about how this plays out on the Apple side.

Note: I've only given this a couple of weeks of playing around in my off time. Thoughts here are first impressions, not expert opinions from a seasoned iOS dev. Take all of the following with a grain of salt.

Swift

Apple's programming language for their products is fairly quick to pick up. The tour of the language on the swift docs site gets you most of the way there, and most of it is analogous to any other language that supports classes.

Unwrapping

The one concept that takes getting used to is unwrapping. I'll do my best to explain concisely, since I don't want to distract too much from the rest of the post.

Swift has an Optional type that you can denote with a question mark. If I were to try to use this value later, I would get a compile time error:

let color: String? = nil

print(color) // Warning: Expression implicitly coerced from 'String?' to 'Any'

color as a variable has reserved space in memory for a value, but it's not guaranteed that the value is there. That is what the Optional type is communicating. Hence the warning when print receives an optional type.

The swift way of handling this is to unwrap the value.

if let color = color {
    print(color)
}

On the left side of the operator, I'm declaring a new name variable for the block scope. On the right, I'm assigning it the value of my original name variable. At the start of this line, I'm checking that a value exists with an if statement.

By this point, if a value exists, Swift knows this is no longer an Optional value and we are safe to call print with color.

There's even a shorthand since this is so commonly done:

if let color {
    print(color)
}

Essentially, unwrapping is a fancy term for asserting that a value exists on the variable. The nice thing about swift as a statically typed language is that it will encourage you to check your values in ambiguous scenarios.

XCode

XCode has been interesting to use. Unsurprissingly, the typography and design is very Apple. In that sense, it's delightful.

Like any new program, it's easy to be overwhelmed by the sheer number of menus and options to toggle. However, it doesn't take long to find the few options you'll use the most often.

It's a seamless dev experience working with iOS apps here. Unsurprisingly, when the software is designed for a specific platform, it works really well with said platform. Spinning up a simulated app environment is quick and easy. UI wise, you have a couple of options between SwiftUI and UIKit when sticking to Apple support. And adding components from those libraries is as easy as drag-and-drop.

One thing that takes getting used to is a UI interface for adjusting any elements I add to the Storyboard visual editor. When you're used to scanning documentation or skimming through the options in intellisense to find input options, it feels a bit slow and laborious to have to paw through menus to find where I can resize a button element. Perhaps there are other ways of working with elements that are more text driven. Either way, I'm sure it would just take some time to familiarize oneself with where all the menus are.

One of the most interesting features lies in the intersection between Interface Builder elements and the hand written code. Interface Builder is the WYSIWYG style visual pane for adding elements to a view. However, you can connect them to your own custom written swift classes. Doing so even has the nice UI flare of simply dragging and dropping the element into a line of swift code.

import UIKit

class DetailViewController: UIViewController {
    // This line was added by dragging and dropping (while holding ctrl)
    // the imageView from the Interface Builder
    @IBOutlet var imageView: UIImageView!
    var selectedPhoto: String?
    var photoTitle: String?

     override func viewDidLoad() {
        super.viewDidLoad()
        title = photoTitle

        if let imageToLoad = selectedImage {
            imageView.image = UIImage(named: imageToLoad)
        }
    }
}

There is a certain magic there. Seeing a visual element and being able to hook into it this way.

Impressions

My assumption was that learning Swift would take a bulk of the on ramp time, and then developing on Apple's platform would be smooth sailing. I found the opposite to be true. Swift, at least at this stage in my tinkering, wasn't event the meat and potatoes of my development. It was largely working with interface building, laying out elements there, and then writing a few lines of swift code to get specific functionality.

Perhaps that's unsurprising at the entry level. But I'll say, since a bulk of the benefit of native development is using native interfaces and components, it makes sense that I don't have to type nearly as much boilerplate or initialization code to get started. So in that sense, the language is not the barrier to entry.

However, gettinng up to speed on platform development does take time here. I've come to find this more true with the more tooling I use. Languages are largely similar in their feature set. The frameworks and platforms, however, all require you to learn their way of developing. It's as true for iOS dev as it is for React. So, when learning this sort of thing, much of the time will be spent here.

Largely, though, once that hurdle is covered, the all-encompassing IDE is a smooth experience. Apple is known for excellent design. So it's nice to know that, out of the box, I'm working with components that are elegant to use.

All in all, a great first impression.

A great write up by Dave Thomas on how the title "Software Engineer" simply doesn't cut it when it comes to sucinctly describing what the people in these roles provide:

[A] good developer can sometimes manage to deliver that value without actually writing a line of code. Developers occupy a unique position in most companies, sitting at the confluence of many business units and their customers. Developers often have a broader picture of how the company works and how things interact that many of the business’ managers. Many times I’ve seen a manager deliver a requirement to a team, only to have the team respond, “we can do that, but why not just…?”

So, if we’re engineering anything, it’s value, not software...

The people who deliver value by iteratively refining software deserve to have a name for what they do. It isn’t programmer, designer, analyst, front-end developer, or software engineer. It’s bigger than that, and it’s more subtle.

A great reminder at a time where the "programming" part is becoming more and more automated in our work. The contribution expands beyond the specific tools used to provide solutions.

As kids do, I had a niche affinity. Mine was Smooth Jazz. Houston had a great radio station at the time (95.7 — The Wave.) Any wonder why I decided to pick up playing the saxophone?

The Weather Channel, as you can image, only added fuel to the fire of passion. And Trammell Starks was a frequently played artist.

Here's his collection "Music For Local Forecast."

Certainly very of the genre. The Pat Metheny influence is unmistakeable. There are moments where it's not so smooth. "Jumpin'" in particular has the pianist slamming on the keyboard! (I wonder if this is Starks himself?)

For more fun: TWC Classics lists all the songs played on The Weather Channel from 1983-2002. Trammell had a great showing in '97!

Listen on Youtube

Days I knew as happy, sweet, sequestered days~

Been trying out 3D graphics in Blender! Donut courtesy of Andrew Price's tremmendous material.

Opinions around testing in software are varied. Do you go with the Pyramid shape or the trophy approach? Do you adopt Test Driven Development or lean more heavily on a few critical end-to-end tests after the code is written?

This past month, I spent time working on a side project that explores all the tests working hand-in-hand. I wanted the ideal scenario: 100% test coverage with a full suite of unit, integration, and e2e testing. Let's see if the juice was worth the squeeze!

The App

My target for testing is this upsert page for a client management portal. We'll focus on a simplified version of this page, utilizing React Hook Form to control the form, Mantine to style, and Zod for client side validation.

export default function ClientUpsert({
  client = undefined,
  newClient = false
}: {
  client?: Client;
}) {
  const router = useRouter();
  const {
    register,
    handleSubmit,
    formState: { errors, dirtyFields, isSubmitting, disabled, isValid },
    control
  } = useForm<Client>({
    values: client,
    resolver: zodResolver(ClientUpsertSchema.partial()),
    mode: 'onBlur'
  });


  const onSubmit: SubmitHandler<Client> = async (data) => {
        // Truncating submit logic for brevity's sake.
        upsertClient(id, data);
  };

  return (
    <Container>
      <h1 role="heading">New Client</h1>
      <form onSubmit={handleSubmit(onSubmit)} key="upsert-client">
        <h2 role="heading">{client?.calculated?.fullName}</h2>
        <Stack gap="lg">
          <TextInput
            {...register('profile.firstName', { required: true })}
            label="First Name"
            error={errors.profile?.firstName?.message as string | undefined}
            role="textbox"
          />
          <TextInput
            {...register('profile.lastName', { required: true })}
            label="Last Name"
            error={errors.profile?.lastName?.message as string | undefined}
          />

          <Button
            value={true}
            type="submit"
            role="button"
            disabled={isSubmitting || disabled || !isValid}
          >
            Submit
          </Button>
        </Stack>
      </form>
    </Container>
  );
}

On the API side, I've created a function to upsert the client to MongoDB:

'use server';

/**
 * @throws Will throw an error if the patch object is empty after parsing.
 */
export async function upsertClient(
  id: string,
  patch: Partial<Client>
): Promise<UpdateResult> {
  const parsedPatch = deepPartialify(ClientUpsertSchema).parse(patch);
  if (!parsedPatch || Object.keys(parsedPatch).length === 0) {
    throw new Error('No patch data found');
  }

  const dotNotationPatch = getDotNotation(parsedPatch);

  const collection = client.db('my-db').collection('clients');

  const result = await collection.updateOne(
    { _id: id ? new ObjectId(id) : new ObjectId() },
    { $set: dotNotationPatch },
    { upsert: true }
  );

  revalidatePath(`/client/${id}`);
  return result;
}

You may notice the dance I'm having to do with getDotNotation before sending the patch object. More on that in this blog post. I'm also using a function to make all deeply nested values in my Zod schema optional with deepPartialify(). More on that here.

For the most part, though, pretty straightforward server work. I'm validating the data on the server and then passing data to MongoDB. Because I'm using Next.js, I can add the 'use server'; directive so that I can call this function directly as a server action on the client.

Server Tests

Let's start writing tests for the API:

import {
  upsertClient
} from '../client';
import { Db, MongoClient, ObjectId } from 'mongodb';
import { ObjectIdClient, Client } from '../schema';

declare global {
  var __MONGO_URI__: string;
  var __MONGO_DB_NAME__: string;
}
let mockClient: MongoClient;
let mockDb: Db;

beforeAll(async () => {
  mockClient = await MongoClient.connect(global.__MONGO_URI__, {});
  mockDb = mockClient.db(globalThis.__MONGO_DB_NAME__);
  (client.db as any as jest.Mock) = jest.fn().mockReturnValue(mockDb);

  await mockDb.collection('clients').insertMany([clientOne, clientTwo]);
});
describe('upsertClient', () => {
  const patchId = '677578204c102d057aa44812';

test('should insert a new client with valid patch data', async () => {
    const patch = { profile: { firstName: 'John', lastName: 'Doe' } };

    const result = await upsertClient(
      patchId,
      patch as Partial<Client>
    );

    expect(result).toEqual({
      acknowledged: true,
      upsertedId: new ObjectId(patchId),
      matchedCount: 0,
      modifiedCount: 0,
      upsertedCount: 1
    });

    const expectedPatchClient = {
      _id: new ObjectId(patchId),
      ...patch
    };

    const patchedClient = await mockDb
      .collection('clients')
      .findOne({ _id: new ObjectId(patchId) });

    expect(patchedClient).toEqual(expectedPatchClient);
  });

  test('should update a client with valid patch data', async () => {
    const patch = { profile: { firstName: 'Jim' } };

    const result = await upsertClient(
      patchId,
      patch as Partial<Client>
    );

    expect(result).toEqual({
      acknowledged: true,
      upsertedId: null,
      matchedCount: 1,
      modifiedCount: 1,
      upsertedCount: 0
    });

    const expectedPatchClient = {
      _id: new ObjectId(patchId),
      profile: { firstName: 'Jim', lastName: 'Doe' }
    };

    const patchedClient = await mockDb
      .collection('clients')
      .findOne({ _id: new ObjectId(patchId) });

    expect(patchedClient).toEqual(expectedPatchClient);
  })
};

Lot's going on here, but I'll summarize. These two tests are validating the expected behavior:

1. If a client is found, add the patch data.
2. If not, create a new client.

Additionally, I'm making use of the jest-mongodb preset to spin up a local instance of mongodb. This allows me to have a test db available with every run of Jest. Very handy!

Once I've verified my tests are passing, I'm ready to move on to the client.

Client Tests

There are several tests I could write for my ClientUpsert component. For demonstration, I'll focus on verifying a successful upsert:

/**
 * @jest-environment jsdom
 */

import React from 'react';
import {
  fireEvent,
  screen,
  waitFor
} from '@testing-library/react';
import { renderWithMantineProvider } from '@/lib/test-util/renderWithMantineProvider';
import { componentWithToastify } from '@/lib/test-util/renderWithToastify';
import '@testing-library/jest-dom';

jest.mock('@/lib/api/client/client', () => ({
  __esModule: true,
  upsertClient: jest.fn(),
}));

jest.mock('next/navigation', () => ({
  __esModule: true,
}));


import ClientUpsert from './ClientUpsert';
import { upsertClient } from '@/lib/api/client/client';

describe('<ClientUpsert/ >', () => {

  test('Should show successful update notification after client updated', async () => {
    (upsertClient as jest.Mock).mockReturnValue({
      acknowledged: true,
      upsertedId: null,
      matchedCount: 1,
      modifiedCount: 1,
      upsertedCount: 0
    });

    // Arrange
    renderWithMantineProvider(
      componentWithToastify(<ClientUpsert client={clientOne} />)
    );
    const firstName = screen.getByLabelText('First Name');
    const submit = screen.getByRole('button');

    // Act
    fireEvent.change(firstName, {
      target: { value: 'Big Tuna' }
    });

    // Wait for the form state to update
    await waitFor(() => {
      expect(submit).toBeEnabled();
    });
    fireEvent.click(submit);

    // Assert
    expect(await screen.findByText('Client Updated')).toBeInTheDocument();
  });
});

Here I'm implmenting a three-phased test. I want this to be a true unit test of the component, so I'm going to mock the upsertClient call. We'll handle seeing the two interact in my end-to-end test.

The nice thing about doing so is that, should the logic of this component be used elsewhere with another method passed to the onSubmit, I don't have to worry about writing a whole new set of tests for it. We can simply focus on the UI performing as it should.

To stay true to the philosophy of React Testing Library, I'm also only concerned with seeing the visual sign of a successful upsert. In this case, it's a toast message appearing on the page.

With tests passing, it's time for end-to-end testing!

End-To-End Testing with Playwright

Playwright is my flavor of the month for this project, and it's been a great experience so far! I'll skip the many quality-of-life features and will focus on finishing out our task here.

Let's write our user flow:

import test, { expect } from '@playwright/test';
import { randomUUID } from 'node:crypto';

test.describe('New Client', () => {
  const clientInput = {
    firstName: 'Test',
    lastName: randomUUID(),
  };

  const clientFullName = `${clientInput.firstName} ${clientInput.lastName}`;


  test('should upsert a new client', async ({ page }) => {
    await page.goto('./client/new');
    expect(page.getByRole('heading', { name: 'New Client' })).toBeVisible();

    await page.getByLabel('First Name').fill(clientInput.firstName);
    await page.getByLabel('Last Name').fill(clientInput.lastName);

    const submitButton = page.getByRole('button', { name: 'Submit' });

    expect(submitButton).toBeEnabled();
    submitButton.click();

    await expect(
      page.getByRole('heading', {
        name: clientFullName
      })
    ).toBeVisible();

    await page.goto('./clients');
    await expect(page.getByText(clientFullName)).toBeVisible();
  });
});

Reading the code, the user journey is being described almost in plain english. I'm navigating to the page, filling out the form, submitting, and then verifying on a page containing all clients that the new document exists.

This can be tricky to maintain. Depending on how you chose to target elements, you may find yourself needing to update your selectors with any change. Targeting by element role is a safe approach because we are viewing essential page elements from the perspective of the user. And not just the typical mouse-in-hand user, but for screen reader users as well!

Evaluating Time & Testing Your Deliverable

If I were to average out the amount of time spent trying to overachieve as a tester, I would say I spent nearly half of my time developing and half writing the test suite.

In some projects, that might not be a problem. However, I can see where doing this yielded diminishing returns after a certain point. Hypothetically, this scenario of a single use page likely only needed an end-to-end test, perhaps with integration tests for my API endpoints that will likely see reuse.

Yet, in another project, if this took two devs, one on the front end, one on the back end, I would want both of them delivering tests for their portions.

At the end of the day, like any other choice in software, the answer to "what to test" is "it depends." Each application is different in scope, technology, and usage. A reusable widget in an iFrame can likely benefit more from unit testing, while your client sign up flow will need e2e testing. It's really dependent on the size of your deliverable, then.

Not to say that if you have e2e coverage, you can skip over integration and unit tests. It could be a headache breaking those components out as modular units if that's the case.

Ultimately, though, time is a limited resource. Testing itself is worth the investment of time for several reasons: confidence in the functionality of your app, thoughtfulness around edge cases, and communication of it's expected behavior. Where you chose to invest that time, however, is dependent on your highest priority outcome.

Supplemental Reading

• Alex Gusman has a great walk through of his own experience using playwright and preferring e2e testing.
• Many of Kent C. Dodd's articles on testing have informed my thoughts here. Most relevant may be How To Know What To Test.
• I've been reading Mark Richards' and Neal Ford's Fundamentals of Software Architecture. They preface that any technical conversation starts off with "It Depends." So this article is just another big "It Depends" conversation.
• Speaking of, I love this faux O'Reilly cover also called "It Depends."

From Justin Weiss on how your tests should work in phases:

1. First, you set some stuff up (“Arrange”)
2. Then, you do something (“Act”)
3. Then, you make sure that what you expected to happen, actually happened. (“Assert”)

Lovely. Flexible enough to accomodate multiple levels of complexity, while giving enough of a framework to get you started on the path to testing quickly.

Kyle MacLachlan wrote a beautiful piece in memory of David Lynch for The New York Times:

Though my lifelong friend, collaborator and mentor David Lynch was as eloquent as anyone I’d ever met — and a brilliant writer — he was not necessarily a word person...

How could words possibly do justice to an experience like that?

It’s why David was not just a filmmaker: He was a painter, a musician, a sculptor and a visual artist — languageless mediums.

When you are outside language, you are in the realm of feeling, the unconscious, waves. That was David’s world. Because there’s room for other people — as the listeners, the audience, the other end of the line — to bring some of themselves.

To David, what you thought mattered, too.

I'm dumbfounded by how there is no correlation between how skilled you are and how much enrichment you get out of practicing an art form.

I spent years in strict training to achieve a certain level of performance ability in music, under an assumption that fulfillment comes from mastery. And don't get me wrong — there is a certain freedom in being able to smoothly skate across an instrument.

But it's not a requirement. And the joy can come from day 1.

The real source is flow. There are plenty reasons to do creative work, but I'd say most of us find the magic in those moments where time slips away.

There are several ingredients to reaching that state. When it comes to ability, though, the key lever is how well the task in front of you matches your skill level.

Is it challenging enough where you're doing a bit of reaching?

Is it easy enough where you don't feel totally overwhelmed by what you're doing?

That's it. And it's relative.

Saxophone is my most fluent instrument. To achieve flow, I would have to be working on a new piece with significant technical and expressive challenges to get there. But when I'm playing guitar, I find it just by spending time moving between two chords again and again and again.

Especially as we're getting started, we're moved by the product of creative work. Someone's ability as a performer inspires us, or a stunning painting really grabs us. That's great for lighting the spark.

If that's the only source of fuel in the practice, though, there's this resentment of not being at that ability.

Most of the fulfillment has to come from the actual act of doing the thing. Thankfully, a certain level of skill is not required. Just flow.

Listen on Youtube

Star eyes, flashing eyes in which my hopes rise
Let me show you where my heart lies
Let me prove that it adores
That lovingness of yours

🤩

Enjoying this lovely arrangement with Helen O'Connell & Jimmy Dorsey.

💎 🌊

Inspired by the fantastical work of ELIOLI.