Another cURL: This time validating TLS versions

I had to demonstrate that our hosting of a single page app did not accept TLS version 1.1,

This is the proof

curl -o /dev/null -s -w "%{http_code}\n" https://koala-moon.com --insecure --tls-max 1.1
Returns: 0000

curl -o /dev/null -s -w "%{http_code}\n" https://koala-moon.com --insecure --tls-max 1.2
Returns: 200

Easy eh?

API development: Lazy tip # 25

TL;DR;

Use cURL and pipe to json_pp “| json_pp

So for the longest time when working on API’s my process would look like this:

  • Write code in Visual Studio Code
  • Change to Postman and click send (try again as Nodemon has not finished restarting)
  • Check result
  • Change back to Visual Studio Code
  • Repeat

But now its

  • Write code in Visual Studio Code
  • In the Visual Studio Code terminal press ‘up arrow’ then ‘return’
  • Check result
  • Repeat

This has speeded up my development and reduced frustration.

How?
In the terminal I make use of cURL and append the command with | json_pp

For example

curl --location --request POST 'localhost:61100/my-service/report/progress' \
--header 'Authorization: <token> \
--header 'Content-Type: application/json' \
--data-raw '{"ids":["5d765d10a13c116f3ce44d04","62bc1433eb87da610a436a48","62bc7a1147f2e0f3854aa268","62bc2dd747f2e04f724a9e26"]}' | json_pp

Top tip:

Use postman on the first iteration, then use it to export cURL command

Fire and Forget

Yesterday was spent diagnosing what was reported as CORS issue in a Node service used for logging that was being called by front end apps.

CORS turned out to be a red herring, it ended up being the wrong format for a new database connection string, less than gracefully handling errors (it would just die and return 500), but what concerned me was the frontend request (sending a packet to a remote logging service) was not necessary to the successful sequence of editing and saving an object (plus it was sent prior to the event being successful) and would halt following requests.

The frontend fix was a “fire and forget” call to send a logging packet to the Node service, while not ideal if the service was down it would not effect frontend users experience.

const logEvent = (str) => {
  const send = async (str) => {
    // send the request, it might throw an error
    throw new Error(`thrown....${str}`)
  }

  send(str).catch(() => {
    // If you don't catch the error you can/will end up with a non handled exception error.
    console.log('It did not break, log error with service like Azure App Insights')
  })
}

console.log(
  logEvent("do nothing")
)

NPM update

So, for a long time now our projects CI/CD pipelines have an audit step and will prevent merging if there and high risk issues. Nothing complicated just

npm audit

Then a developer would need to sit down and follow their own approach to updating NPM packages. Mine was to install all the patch versions – test, minor versions and test. Then I’d take a stab at any major versions after reading any release notes.

See: https://docs.npmjs.com/about-semantic-versioning

It works, but it is a pain.

Then I found NPM GUI

https://www.npmjs.com/package/npm-gui

After installing it globally, I run the following command

npm-gui

This opens a browser tab and will first show all the global packages and versions. Navigate to your directory that has the package.json file, a nice table of installed node packages is displayed with required, current, compatible and latest versions. Clicking the versions will install them.

I still follow the same work flow of patches, minor and major versions but it takes me far less time and cognitive load to update projects.

It might not be perfect, but the rest of my team have started using this and its helped overcome the dread of package maintenance.

More regEx in VSCode

I had a list of a few thousand words either 3, 4 or 5 letters, i needed to turn the imported list into an array.

const listFourLetterWords = [
ages
foot
good
hope
host
]

Using VSCodes finder with regEx it took moments to turn the 1000’s of words into

const listFourLetterWords = [
"ages",
"foot",
"good",
"hope",
"host",
]

Solution:

Find: \b[a-zA-Z]{4}\b

Replace: “$&”,

Typescript and Mongoose (MongoDb) Discriminators

Many thanks to Manuel Maldonado and his excellent article https://hackernoon.com/how-to-link-mongoose-and-typescript-for-a-single-source-of-truth-94o3uqc. This post carries on from where he left off.

Discriminators – the missing step.

The current project at time of writing makes use of Mongoose’s schema inheritance mechanism. That enables you to have multiple models with overlapping schemas on top of the same underlying MongoDB collection. In other words a single collection that can have multiple models. Mostly this is a cost saving exercise. For more detail see.

https://mongoosejs.com/docs/discriminators.html

https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-mongoose#using-mongoose-discriminators-to-store-data-in-a-single-collection

How to add types to discriminators

Begin by reading Manuel’s article first!

You will see how to set up models, schemas, statics and methods that are strongly typed and play well with your IDE IntelliSense.

You are using Typescript?

You project is using Typescript and you have installed the following package?

npm i @types/mongoose

https://www.npmjs.com/package/@types/mongoose

Discriminators and Typescript

Here is the finished Base Model:

import { model, Schema } from "mongoose";

const collection = "GeneralPurpose";

const baseOptions = {
  discriminatorKey: "__type",
  collection,
  timestamps: true,
};

export const Base = model("Base", new Schema({}, baseOptions));

Here is the Base Model being used as the inherited model:

import { Document, Model, Types, Schema } from "mongoose";
import { Base } from "./Base.model";

export const MODEL_REF = "JournalBlock";

export interface JournalBlock {
  // _id?: string; // Note, _id is inherited from Document type
  blockTitle: string;
  icon: string;
  createdAt: string;
  updatedAt: string;
  parentId: string;
}

export interface JournalBlockDocument extends JournalBlock, Document {
  minify(): unknown;
}

export interface JournalBlockModel extends Model<JournalBlockDocument> {}

export const JournalBlockSchema: Schema = new Schema({
  blockTitle: { type: String, required: [true, "A title is required"] },
  icon: { type: String, required: [true, "A URL for the icon is required"]},
  introText: {
    type: String,
    required: [true, "A introduction text required"],
    },
  },
});

// Just to prove that hooks are still functioning as expected
JournalBlockSchema.pre("save", function () {
   console.log("PRE SAVE", this);
 }).post("delete", function () {
   console.log("post delete", this);
});

// Add a method. In this case change the returned object
JournalBlockSchema.methods.minify = async function (
  
  this: JournalBlockDocument

) {
  const response: JournalBlock & { _id: string } = {
    _id: this._id,
    icon: this.icon,
    introText: this.introText,
    createdAt: this.createdAt,
    updatedAt: this.updatedAt,
    parentId: this.parentId,
  };
  return response;
};


// This is the magic where we connect Typescript to the Mongoose inherited base model (discriminator)

export const JournalBlockModel = Base.discriminator<
  JournalBlockDocument,
  JournalBlockModel
>(MODEL_REF, JournalBlockSchema);

Use

Import the model into for code as normal. Your IDE will predict the returned Documents from queries like “.findById” including any additional methods or statics

JS Sorting with .localCompare

I’ve been using Javascript for 2 years now and can not remember ever having to sort collections. 95% of what I am doing is backend services work and I try to off load as much data manipulation to the databases, but this week I have had 7 instances of having to sort data.

This caught me out:

For a given collection:

const collection = [ {name: "Dave"}, {name: "Donna"}, {name: "dave"}, {name: "Derek"}, {name: "Dave"},];

Sorted with:

collection.sort(
  (a,b) => { 
    if (a.name < b.name) return -1; 
    if (a.name > b.name) return 1; 
    return 0;
  })

Returns this:

[ { name: 'Dave' },  { name: 'Dave' },  { name: 'Derek' },  { name: 'Donna' },  { name: 'dave' } ]

Which is not what I need.

.localCompare to the rescue

collection.sort(
  (a, b) => a.name.localeCompare(b.name)
)

Returns this:

[ { name: 'dave' },  { name: 'Dave' },  { name: 'Dave' },  { name: 'Derek' },  { name: 'Donna' } ] 

But there is more

.localCompare method has additional options. For the same collection above, this

collection.sort(
  (a, b) => a.name.localeCompare(b.name, 'en', { sensitivity: 'base' })
)

Returns this:

[ { name: 'Dave' },  { name: 'Dave' },  { name: 'dave' },  { name: 'Derek' },  { name: 'Donna' } ] 

Be aware that full implementation of .localCompare in Node.js is dependant on the version you are running.

JSTypescript Debounce

Debounce simply controls how many times a function to be executed over time.

For example, when typing into a search box, you probably don’t want to be calling an external API every time the user presses a key, it is more likely you would wait until they have paused or have finished entering text. Then after second or 2 fetch the data.

If you run the example below f() is called multiple times

let i = 0;
const f = () => {
  console.log(`foo ${++i}`);
};
f(); // foo 1
f(); // foo 2
f(); // foo 3
f(); // foo 4
f(); // fo0 5

console.log(i); // 5

Now wrapping the function inside a debounce method only the last call is completed

const debounce = (
  fnc: (...params: any[]) => unknown,
  n: number,
  immediately: boolean = false
) => {
  let timer: number | undefined = undefined;
  return function (this: any, ...args: any[]) {
    if (timer === undefined && immediately) {
      fnc.apply(this, args);
    }
    clearTimeout(timer);
    timer = setTimeout(() => fnc.apply(this, args), n);
    return timer;
  };
};

let i = 0;
const f = debounce(() => console.log(`foo ${++i}`), 2000);
f(); 
f();
f();
f();
f(); // foo 1

console.log(i) // 0 

To recap; If a user keeps clicking on a button it is only after there has been a pause will the last click be carried out.

Throttling, is very similar to debounce but this time it is the FIRST click that is actioned and subsequent clicks are ignored until a period of time has passed.

Apollo Federation with local schemas

Problem:

We have an eco system of 12+ micro services using either .net or node and each exposing a restful API.

I built my first Apollo GraphQL solution in early 2019 and it works really well, the down side its huge and a pain to maintain as it basically is a collection of types, resolvers and requests to the different services. So we went with federation of the different services.

Now before I begin there is an excellent article here about using local schemas with Apollo Federation and Gateway.

This works but we needed a different approach that allowed us to slowly increment changes and separate out services. We also had the issue that a couple of the services could not be touched (this was a commercial decision).

Solution:

import { ApolloServer } from 'apollo-server-express';
import { ApolloGateway, RemoteGraphQLDataSource, LocalGraphQLDataSource } from '@apollo/gateway';
import { buildFederatedSchema } from '@apollo/federation';
import { DocumentNode } from 'graphql';
import depthLimit from 'graphql-depth-limit';
import userService from './services/user';
import organisationService from './services/organisation';

const remoteServices = {
  Goals: {
    url: 'https://api.mysite.com/goal/graphql',
  },
  Curriculum: {
    url: 'https://api-mysite.com/curriculum/graphql',
  },
};

const localServices = {
  ...userService,
  ...organisationService,
};

const services = ({
  ...localServices,
  ...remoteServices,
} as unknown) as {
  [index: string]: {
    url?: string;
    schema: DocumentNode;
  };
};

const DUMMY_SERVICE_URL = 'localService';

const TOKEN = process.env.KEY || ''; // This string is uses to access the remote services inside or micro service environment. It is used once when the service starts. All other queries and requests us the bearer token supplied by the client
 
    const gateway = new ApolloGateway({
  serviceList: Object.keys(services).map((name) => ({
    name,
    url: services[name].url || DUMMY_SERVICE_URL,
  })),
  __exposeQueryPlanExperimental: false,
  buildService({ name, url }) {
    if (url === DUMMY_SERVICE_URL) {
      return new LocalGraphQLDataSource(buildFederatedSchema(services[name].schema)); // The collection of local services
    } else {
      return new RemoteGraphQLDataSource({
        url,
        willSendRequest({ request, context }) {
          request.http?.headers.set('authorization', context.token || TOKEN); // This is the clients bearer token
        },
      });
    }
  },
});

// Just a regular Apollo Server. Its exported and used in the server/app file
export const gqlServer = new ApolloServer({
  gateway,
  subscriptions: false,
  validationRules: [depthLimit(8)],
  engine: false,
  context: ({ req }) => {
    const authorization = req.headers.authorization || '';
    return { authorization };
  },
  plugins: [
    {
      serverWillStart() {
        console.log('Server starting up!');
      },
    },
  ],
});

What the above allows us to do is create local schemas like this. You can always separate this into different files.

import { gql } from 'apollo-server-express';
import ConfigManager from '../../../lib/configSource/ConfigManager';
import axios from 'axios';
import { IAuthContext, makeHeaders, Error401, NotFound, errorHandling200 } from './utils'; // Common interfaces, a method to create a request header
import errors from '../../../lib/errors';

const BASE_URL = process.env.userProfileAPI;

interface User {
  id: string;
  firstName: string;
  lastName: string;
  displayName: string;
  lastLoggedIn: string;
  roles: string[];
  organizationId: string;
}

const typeDefs = gql`
  type User @key(fields: "id") {
    id: ID!
    email: String
    firstName: String
    lastName: String
    displayName: String
    lastLoggedIn: String
    roles: [String]
    organizationId: String
    organisation: Organisation
  }

  type NotFound {
    message: String
  }

  type Error401 {
    message: String
  }

  union UserResult = User | NotFound | Error401

  extend type Organisation @key(fields: "id") {
    id: ID! @external
    users: [UserResult]
  }

  extend type Query {
    User(id: ID!): UserResult
  }
`;

const resolvers = {
  Query: {
    User: async (
      parent: unknown,
      { id }: { id: string },
      ctx: IAuthContext,
    ): Promise<null | User | NotFound | Error401> => {
      try {
        const { data } = await getUser(id, ctx);
        return {
          __typename: 'User',
          ...data,
        };
      } catch (error) {
        return errorHandling200(error, error.response.status, `User with id "${id}" not found`);
      }
    },
  },
  User: {
    async __resolveReference(user: { id: string }, ctx: IAuthContext): Promise<User | NotFound | Error401> {
      try {
        const { data } = await getUser(user.id, ctx);
        return {
          __typename: 'User',
          ...data,
        };
      } catch (error) {
        return errorHandling200(error, error.response.status, `User with id "${user.id}" not found`);
      }
    },
    async organisation({ organizationId }: { organizationId: string }): Promise<unknown | null> {
      return { __typename: 'Organisation', id: organizationId };
    },
  },
  Organisation: {
    async users(parent: { id: string }, _: unknown, ctx: IAuthContext): Promise<User[] | NotFound | Error401> {
      try {
        const users = await getUserByOrganisation(parent.id, ctx);
        return users.map(user => ({
          __typename: "User",
          ...user
        }));
      } catch (error) {
        return errorHandling200(error, error.response.status, `User with id "${parent.id}" not found`);
      }
    },
  },
};

export default {
  user: {
    schema: {
      typeDefs,
      resolvers,
    },
  },
};

Quick1: JS funky date requirement

The problem is:

Take a date object, set the time to 09:15 the following day. If that day is a Saturday, Sunday or Monday bump the date to the next Tuesday.

Solution:

const reminderTime = (date: Date): Date => {
  const response = new Date(date);
  
  const SUNDAY = 0;
  const MONDAY = 1;
  const TUESDAY = 2;
  const SATURDAY = 6;
  
  response.setDate(response.getDate() + 1);
  
  if ([SUNDAY, MONDAY, SATURDAY].some((x) => x === response.getDay())) {
    response.setDate(response.getDate() + ((TUESDAY + 7 - response.getDay()) % 7));
  }

  response.setHours(9);
  response.setMinutes(15);

  return response;
};

Is the date local or UTC? That depends on how the Date object passed into the method was created. In either case local or UTC is preserved.

Here are some proofs:

console.log(reminderTime(new Date(2020, 10, 2))); 
// Returns Tue Nov 03 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 3)));
// Returns Wed Nov 04 2020 09:15:00 GMT+0000 (Greenwich Mean Time)

console.log(reminderTime(new Date(2020, 10, 4))); 
// Returns Thu Nov 05 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 5)));
// Returns Fri Nov 06 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 6)));
// Returns Tue Nov 10 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 7)));
// Returns Tue Nov 10 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 8)));
// Returns Tue Nov 10 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 10, 9)));
// Returns Tue Nov 10 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 11, 31)));
// Returns Fri Jan 01 2021 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 1, 29)));
// Returns Tue Mar 03 2020 09:15:00 GMT+0000 (Greenwich Mean Time) 

console.log(reminderTime(new Date(2020, 1, 30)));
// Returns Tue Mar 03 2020 09:15:00 GMT+0000 (Greenwich Mean Time)