Postman and Faker

I am a huge fan of Postman when developing API’s. I dont think i use even 20% of its abilities.

Today I found out that the Postman Pre-request scripts also have access to Faker.

Here is an example dynamic request body for creating a user.

const randomInt = Math.floor((Math.random()*100 +1));

const firstName = pm.variables.replaceIn("{{$randomFirstName}}");
const lastName = pm.variables.replaceIn("{{$randomLastName}}");
const domain = pm.variables.replaceIn("{{$randomCompanyName}}").replace(/[, ]+/g, ".").trim();


const body = {
firstName,
lastName,
email: ${firstName}.${lastName}@${domain}.c
om

}

pm.environment.set('body', JSON.stringify(body))

Here is Postman’s list of Faker options

https://learning.postman.com/docs/writing-scripts/script-references/variables-list/

Another cURL: This time validating TLS versions

I had to demonstrate that our hosting of a single page app did not accept TLS version 1.1,

This is the proof

curl -o /dev/null -s -w "%{http_code}\n" https://koala-moon.com --insecure --tls-max 1.1
Returns: 0000

curl -o /dev/null -s -w "%{http_code}\n" https://koala-moon.com --insecure --tls-max 1.2
Returns: 200

Easy eh?

API development: Lazy tip # 25

TL;DR;

Use cURL and pipe to json_pp “| json_pp

So for the longest time when working on API’s my process would look like this:

  • Write code in Visual Studio Code
  • Change to Postman and click send (try again as Nodemon has not finished restarting)
  • Check result
  • Change back to Visual Studio Code
  • Repeat

But now its

  • Write code in Visual Studio Code
  • In the Visual Studio Code terminal press ‘up arrow’ then ‘return’
  • Check result
  • Repeat

This has speeded up my development and reduced frustration.

How?
In the terminal I make use of cURL and append the command with | json_pp

For example

curl --location --request POST 'localhost:61100/my-service/report/progress' \
--header 'Authorization: <token> \
--header 'Content-Type: application/json' \
--data-raw '{"ids":["5d765d10a13c116f3ce44d04","62bc1433eb87da610a436a48","62bc7a1147f2e0f3854aa268","62bc2dd747f2e04f724a9e26"]}' | json_pp

Top tip:

Use postman on the first iteration, then use it to export cURL command

Fire and Forget

Yesterday was spent diagnosing what was reported as CORS issue in a Node service used for logging that was being called by front end apps.

CORS turned out to be a red herring, it ended up being the wrong format for a new database connection string, less than gracefully handling errors (it would just die and return 500), but what concerned me was the frontend request (sending a packet to a remote logging service) was not necessary to the successful sequence of editing and saving an object (plus it was sent prior to the event being successful) and would halt following requests.

The frontend fix was a “fire and forget” call to send a logging packet to the Node service, while not ideal if the service was down it would not effect frontend users experience.

const logEvent = (str) => {
  const send = async (str) => {
    // send the request, it might throw an error
    throw new Error(`thrown....${str}`)
  }

  send(str).catch(() => {
    // If you don't catch the error you can/will end up with a non handled exception error.
    console.log('It did not break, log error with service like Azure App Insights')
  })
}

console.log(
  logEvent("do nothing")
)

NPM update

So, for a long time now our projects CI/CD pipelines have an audit step and will prevent merging if there and high risk issues. Nothing complicated just

npm audit

Then a developer would need to sit down and follow their own approach to updating NPM packages. Mine was to install all the patch versions – test, minor versions and test. Then I’d take a stab at any major versions after reading any release notes.

See: https://docs.npmjs.com/about-semantic-versioning

It works, but it is a pain.

Then I found NPM GUI

https://www.npmjs.com/package/npm-gui

After installing it globally, I run the following command

npm-gui

This opens a browser tab and will first show all the global packages and versions. Navigate to your directory that has the package.json file, a nice table of installed node packages is displayed with required, current, compatible and latest versions. Clicking the versions will install them.

I still follow the same work flow of patches, minor and major versions but it takes me far less time and cognitive load to update projects.

It might not be perfect, but the rest of my team have started using this and its helped overcome the dread of package maintenance.

More regEx in VSCode

I had a list of a few thousand words either 3, 4 or 5 letters, i needed to turn the imported list into an array.

const listFourLetterWords = [
ages
foot
good
hope
host
]

Using VSCodes finder with regEx it took moments to turn the 1000’s of words into

const listFourLetterWords = [
"ages",
"foot",
"good",
"hope",
"host",
]

Solution:

Find: \b[a-zA-Z]{4}\b

Replace: “$&”,

Typescript and Mongoose (MongoDb) Discriminators

Many thanks to Manuel Maldonado and his excellent article https://hackernoon.com/how-to-link-mongoose-and-typescript-for-a-single-source-of-truth-94o3uqc. This post carries on from where he left off.

Discriminators – the missing step.

The current project at time of writing makes use of Mongoose’s schema inheritance mechanism. That enables you to have multiple models with overlapping schemas on top of the same underlying MongoDB collection. In other words a single collection that can have multiple models. Mostly this is a cost saving exercise. For more detail see.

https://mongoosejs.com/docs/discriminators.html

https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-mongoose#using-mongoose-discriminators-to-store-data-in-a-single-collection

How to add types to discriminators

Begin by reading Manuel’s article first!

You will see how to set up models, schemas, statics and methods that are strongly typed and play well with your IDE IntelliSense.

You are using Typescript?

You project is using Typescript and you have installed the following package?

npm i @types/mongoose

https://www.npmjs.com/package/@types/mongoose

Discriminators and Typescript

Here is the finished Base Model:

import { model, Schema } from "mongoose";

const collection = "GeneralPurpose";

const baseOptions = {
  discriminatorKey: "__type",
  collection,
  timestamps: true,
};

export const Base = model("Base", new Schema({}, baseOptions));

Here is the Base Model being used as the inherited model:

import { Document, Model, Types, Schema } from "mongoose";
import { Base } from "./Base.model";

export const MODEL_REF = "JournalBlock";

export interface JournalBlock {
  // _id?: string; // Note, _id is inherited from Document type
  blockTitle: string;
  icon: string;
  createdAt: string;
  updatedAt: string;
  parentId: string;
}

export interface JournalBlockDocument extends JournalBlock, Document {
  minify(): unknown;
}

export interface JournalBlockModel extends Model<JournalBlockDocument> {}

export const JournalBlockSchema: Schema = new Schema({
  blockTitle: { type: String, required: [true, "A title is required"] },
  icon: { type: String, required: [true, "A URL for the icon is required"]},
  introText: {
    type: String,
    required: [true, "A introduction text required"],
    },
  },
});

// Just to prove that hooks are still functioning as expected
JournalBlockSchema.pre("save", function () {
   console.log("PRE SAVE", this);
 }).post("delete", function () {
   console.log("post delete", this);
});

// Add a method. In this case change the returned object
JournalBlockSchema.methods.minify = async function (
  
  this: JournalBlockDocument

) {
  const response: JournalBlock & { _id: string } = {
    _id: this._id,
    icon: this.icon,
    introText: this.introText,
    createdAt: this.createdAt,
    updatedAt: this.updatedAt,
    parentId: this.parentId,
  };
  return response;
};


// This is the magic where we connect Typescript to the Mongoose inherited base model (discriminator)

export const JournalBlockModel = Base.discriminator<
  JournalBlockDocument,
  JournalBlockModel
>(MODEL_REF, JournalBlockSchema);

Use

Import the model into for code as normal. Your IDE will predict the returned Documents from queries like “.findById” including any additional methods or statics

JS Sorting with .localCompare

I’ve been using Javascript for 2 years now and can not remember ever having to sort collections. 95% of what I am doing is backend services work and I try to off load as much data manipulation to the databases, but this week I have had 7 instances of having to sort data.

This caught me out:

For a given collection:

const collection = [ {name: "Dave"}, {name: "Donna"}, {name: "dave"}, {name: "Derek"}, {name: "Dave"},];

Sorted with:

collection.sort(
  (a,b) => { 
    if (a.name < b.name) return -1; 
    if (a.name > b.name) return 1; 
    return 0;
  })

Returns this:

[ { name: 'Dave' },  { name: 'Dave' },  { name: 'Derek' },  { name: 'Donna' },  { name: 'dave' } ]

Which is not what I need.

.localCompare to the rescue

collection.sort(
  (a, b) => a.name.localeCompare(b.name)
)

Returns this:

[ { name: 'dave' },  { name: 'Dave' },  { name: 'Dave' },  { name: 'Derek' },  { name: 'Donna' } ] 

But there is more

.localCompare method has additional options. For the same collection above, this

collection.sort(
  (a, b) => a.name.localeCompare(b.name, 'en', { sensitivity: 'base' })
)

Returns this:

[ { name: 'Dave' },  { name: 'Dave' },  { name: 'dave' },  { name: 'Derek' },  { name: 'Donna' } ] 

Be aware that full implementation of .localCompare in Node.js is dependant on the version you are running.

JSTypescript Debounce

Debounce simply controls how many times a function to be executed over time.

For example, when typing into a search box, you probably don’t want to be calling an external API every time the user presses a key, it is more likely you would wait until they have paused or have finished entering text. Then after second or 2 fetch the data.

If you run the example below f() is called multiple times

let i = 0;
const f = () => {
  console.log(`foo ${++i}`);
};
f(); // foo 1
f(); // foo 2
f(); // foo 3
f(); // foo 4
f(); // fo0 5

console.log(i); // 5

Now wrapping the function inside a debounce method only the last call is completed

const debounce = (
  fnc: (...params: any[]) => unknown,
  n: number,
  immediately: boolean = false
) => {
  let timer: number | undefined = undefined;
  return function (this: any, ...args: any[]) {
    if (timer === undefined && immediately) {
      fnc.apply(this, args);
    }
    clearTimeout(timer);
    timer = setTimeout(() => fnc.apply(this, args), n);
    return timer;
  };
};

let i = 0;
const f = debounce(() => console.log(`foo ${++i}`), 2000);
f(); 
f();
f();
f();
f(); // foo 1

console.log(i) // 0 

To recap; If a user keeps clicking on a button it is only after there has been a pause will the last click be carried out.

Throttling, is very similar to debounce but this time it is the FIRST click that is actioned and subsequent clicks are ignored until a period of time has passed.