Posts

Posted on

Enhancing Focus with Thematic Sprints in Our Dynamic Development Team

Created by DALL·E

In the fast-paced world of software development, maintaining focus is crucial, especially for our dynamic team, which consists of a core of four developers but can expand to about a dozen. We’ve found that implementing thematic sprints—where each sprint is named after a specific theme or goal—significantly enhances our productivity and engagement

When our team is small, usually capped at six members, we experience a focused output with active involvement from developers in the planning phase. However, we’ve noticed that stories can grow in complexity, leading to a loss of focus and extended timelines. To address this, we started naming our sprints with clear themes, creating a shared understanding of our objectives.

This simple practice reinforces our commitment during daily stand-up meetings, where we consistently reference the sprint theme. By framing our work around these themes, developers stay aligned with our goals and feel a greater sense of ownership.

Moreover, thematic sprints foster collaboration. As we engage with a specific focus, discussions naturally emerge, leading to innovative solutions and richer features. For instance, dedicating a sprint to user experience made our team more attentive to feedback, resulting in improved designs.

In conclusion, thematic sprints have transformed our development process, enhancing focus and creativity while nurturing a culture of collaboration and innovation. This approach has proven invaluable in our journey as a dynamic software development team.

Posted on

NPM Dependency Notation – idiots guide

Image from ChatGPT

Dependency notation in the package.json file influences how npm handles version installations, and this affects whether npm install may update the package-lock.json file.

Here’s how it works based on different notations:

Version Notations and Their Meanings:

  1. Exact Version (“4.19.2”):
  • Notation: “4.19.2”
  • Meaning: Install exactly version 4.19.2 of the package.
  • Effect on npm install:
  • When this version is specified, npm will always install version 4.19.2, regardless of whether newer minor or patch versions are available. The package-lock.json will not be updated unless you manually change the version in package.json.
  • Example: If version 4.19.3 is available, npm will not install it.


2. Caret (^) Notation (“^4.19.2”):

  • Notation: “^4.19.2”
  • Meaning: Install any compatible version according to semver rules, meaning any version >=4.19.2 and <5.0.0.
  • Effect on npm install:
  • With ^, npm allows updates to the minor and patch versions, but not the major version. This means if a newer patch version (like 4.19.3) or minor version (like 4.20.0) is available, npm will install it. If an update is installed, the package-lock.json will be updated to reflect the new version.
  • Example: If 4.20.1 is available, npm install will update package-lock.json to install 4.20.1.


3. Tilde (~) Notation (“~4.19.2”):

  • Notation: “~4.19.2”
  • Meaning: Install the most recent patch version that matches the specified minor version, meaning any version >=4.19.2 and <4.20.0.
  • Effect on npm install:
  • With ~, npm will allow updates to the patch version but not the minor version. If a newer patch version is available (e.g., 4.19.3), npm will install it, and the package-lock.json will be updated.
  • Example: If version 4.19.5 is available, npm will install that, but it will not install version 4.20.0.


4. Major Version (“4.19”):

  • Notation: “4.19”
  • Meaning: This implies “4.19.x”, which means install the latest available patch version within the 4.19.x range.
  • Effect on npm install:
  • This is similar to using ~4.19.0 but more permissive. It allows updates within the minor version and to any patch version (e.g., 4.19.2 → 4.19.5).
  • Example: If version 4.19.3 or 4.19.5 is available, npm will install it and update package-lock.json.


Impact on npm install and package-lock.json:

  • Exact version (“4.19.2”): No updates will occur unless you change the version manually in package.json.
  • Caret (^4.19.2) or Major version (“4.19”): Newer patch or minor versions will be installed automatically, and this will update package-lock.json with the exact version.
  • Tilde (~4.19.2): Only patch updates are allowed, and package-lock.json will reflect those updates when they occur.


How This Affects npm install:


If you use npm install (as opposed to npm ci), and the notation in your package.json allows for updates (like ^4.19.2), npm may install a newer version within the range, and as a result, the package-lock.json file will be updated with this new version.

On the other hand, if the package-lock.json specifies a version (say 4.19.2), and your package.json allows updates (^4.19.2), running npm install could still install a newer version (like 4.19.3), which would then update the lock file.

This flexibility is a double-edged sword: it allows for automatic updates of patches and minor versions, but if not managed well, it can lead to differences in installed versions across environments, which is why many teams prefer using npm ci in CI/CD pipelines for consistency.

Posted on

NPM install, ci and audit

Image created by ChatGPT

I needed to take a step back and fully understand this issue so I could explain it clearly to both new and experienced developers. The problem surfaced because our deployment pipelines run npm audit, which became a bottleneck in our process. We kept seeing the same vulnerabilities flagged repeatedly, even though they had been fixed multiple times.

Here we go, npm instal, ci and audit

Here’s an overview of the flow from installing a new package with npm to running npm install or npm ci in a pipeline, along with details on how vulnerabilities may resurface through npm audit.

Flow from Installing a Package to CI/CD

  1. Installing a Package:
  • When you install a new package locally (e.g., npm install package-name), npm adds the package to the node_modules directory and updates your package.json and package-lock.json files (or yarn.lock if you use Yarn).
  • package.json specifies the declared dependencies and their versions.
  • package-lock.json contains the exact versions of the installed packages and their entire dependency tree (including transitive dependencies). This ensures that everyone who installs your project gets the same versions of dependencies.

2. Pushing to Version Control:

  • Once you are satisfied with your code, including the new dependency, you push the changes to version control (e.g., Git). It’s important that both the package.json and package-lock.json files are committed to ensure consistency across environments.


3. Pipeline – npm install vs npm ci:

  • npm install:
    • During a build or deployment pipeline, running npm install will install dependencies based on the package.json and update the node_modules directory.
    • If a package-lock.json file exists, npm tries to install exact versions from the lock file, but if it detects any changes (e.g., new versions of dependencies or conflicts), it may update the lock file. (See dependency notation)
  • npm ci:
    • In a CI/CD pipeline, npm ci is preferred as it is faster and more deterministic.
    • It strictly adheres to the versions specified in package-lock.json. If any discrepancies (such as missing or extra dependencies) are found, the entire node_modules directory is deleted, and the exact dependencies from the package-lock.json are installed.
    • npm ci does not update package-lock.json, making it ideal for CI environments where reproducibility is critical.


4. npm audit:

  • During or after the install process, npm may run npm audit to check for security vulnerabilities in your dependencies. It compares the installed packages against a database of known vulnerabilities and flags any risks.
  • npm audit fix can automatically update vulnerable dependencies to the latest non-breaking versions (as defined by semver).


How Do npm audit Problems Reappear?

  1. Indirect Dependencies (Transitive Dependencies):
  • Most npm packages rely on other packages (dependencies of dependencies), and vulnerabilities often arise in these indirect dependencies.
  • Even if you’ve addressed an issue by updating your direct dependencies, some transitive dependencies may still have unresolved issues. This happens because they may not have yet released a fixed version.


2. New Vulnerabilities Discovered:

  • Sometimes, new vulnerabilities are discovered in packages that were previously considered safe. When npm’s vulnerability database is updated, a previously resolved issue may reappear if it’s related to a newly discovered flaw.


3. Out-of-Date Dependencies:

  • When the package-lock.json or a specific package hasn’t been updated for a while, and a vulnerability was later fixed in a newer version, your audit might flag the outdated dependency.
  • Running npm audit regularly (especially on pipelines) will catch such vulnerabilities, but sometimes an older transitive dependency may bring back the issue.


4. Partial Fixes:

  • Sometimes, packages release partial fixes, where only certain issues are resolved. If the fix doesn’t cover all security concerns, npm audit may still flag the package.


5. Conflicts Between Versions:

  • Certain updates may not be backward compatible with your project’s current environment or with other dependencies. This can lead to situations where you are unable to fully update vulnerable dependencies without breaking something else in your codebase.


Dealing with Persistent npm audit Problems:

  • Explicit Version Control: Sometimes you may have to manually control the versions in package-lock.json by using specific version ranges or resolutions (in tools like Yarn) to enforce the use of patched versions.
  • Selective Fixing: If you know a particular vulnerability doesn’t affect your project (e.g., it only impacts a feature you don’t use), you can audit it with exceptions.
  • Monitor Transitive Dependencies: Regularly check your dependency tree to monitor transitive dependencies and see if any have lagging versions. This can be done using tools like npm ls or through dependency-checking platforms.
Posted on

Azure AppInsights with Nodejs adding operation id to response

I needed to add an additional response header to my requests to aid in tracking errors.

Here is my solution. A middleware that will add the AppInsights Operation ID to the response. The Operation ID is unique to each request so searching for it is simple.

import { Request, Response, NextFunction } from 'express';
import appInsights from '../../lib/appInsights';

export default (req: Request, res: Response, next: NextFunction) => {          
    const { operation } = appInsights.getCorrelationContext();    
    
    res.setHeader('X-Operation-ID', operation.id);
    
    next();
};
Posted on

Development questions…

I’ve been working on compiling a set of questions we should ask at the outset of each new feature. In my experience, teams often become laser-focused on coding and implementing new features without considering the broader context. I believe it’s crucial to be proactive and demonstrate the value of our work to the company, as ultimately, the bottom line is what matters most.

For every new feature, let’s ensure we ask these questions. The responses we gather can then inform the creation of tickets and specifications, ensuring that we’re aligned with both technical requirements and organizational objectives.

AspectQuestions
Top Level User Story– What is the main objective or goal of this feature/change?
Security Concerns– Are there any potential security vulnerabilities or risks associated with this work?
Organizational Policies and Processes– Are there specific policies or processes that need to be adhered to during development?
Value Addition– How does this work enhance the application/product?
– What additional benefits or improvements does it bring?
Justification– Why is this work necessary or important?
– What problem or need does it address?
Proving Worth– How can we demonstrate the impact or value of this work?
– What criteria or metrics can be used to measure its success?
Monitoring– What metrics or indicators can be used to track the performance or usage of this feature/change?
– How will we monitor its effectiveness over time?
Technical Requirements– Are there any specific technical constraints or dependencies that need to be considered?
– What technologies or frameworks should be utilized for this work?
User Experience (UX)– How will this work impact the user experience?
– Are there any usability considerations to be aware of?
Testing and Quality Assurance– What testing strategies will be employed to ensure the quality of the implementation?
– Are there any specific test cases or scenarios that need to be addressed?
Scalability and Performance– How will this work scale as the application grows?
– Are there any performance considerations or benchmarks to meet?
Documentation– What documentation needs to be created or updated as part of this work?
– How will knowledge transfer be facilitated for other team members?
Deployment and Rollout– What is the deployment plan for this work?
– Are there any rollout or release strategies to consider?
Feedback and Iteration– How will feedback be collected and incorporated into future iterations?
– What mechanisms are in place for continuous improvement?
Collaboration and Communication– How will communication be maintained between team members and stakeholders throughout the process?
– Are there any collaboration tools or platforms to be used?
Risk Management– What potential risks or challenges could arise during implementation, and how will they be mitigated?
– Is there a contingency plan in place for unexpected issues?

Posted on

Replacing ObjectId with a string in JSON. Using RegEx

Problem: I have a data dump of a MongoDb query in a JSON file. I need to replace the ObjectID(“12345677abc”) with “12345677abc”.

Using Visual Studio Code’s find and replace

Find: 

ObjectId\("([0-9a-fA-F]{24})"\)

Replace with: “$1”

Turns this

"_id" : ObjectId("5e3b1890e032d225a091d43f"),
"userId" : ObjectId("65ed1c2c-922c-4c82-b5bc-7324f69eea10"),

To this

"_id" : "5e3b1890e032d225a091d43f",
"userId" : "65ed1c2c-922c-4c82-b5bc-7324f69eea10",


Bonus:

ISODate\("([^"]+)"\)
Posted on

Mastering New Software with Ease: The Three “Software Whisperer” Principles

Introduction:
Have you ever wondered how some people effortlessly grasp new software applications with minimal training? I was intrigued by this ability and decided to explore the secrets behind their success. Through my journey, I discovered three powerful principles that transformed my approach to learning new software. In this blog post, I’ll share my experience applying the three “Software Whisperer” principles that have made me a more adept learner in the digital landscape.

Embracing the Intuitive Explorer:
The first principle I adopted was the fusion of intuition and curiosity as a powerful learning tool. I began trusting my instincts and letting my inquisitiveness guide me as I delved into new applications. Embracing various approaches and adapting my mindset based on my discoveries proved highly effective. With each encounter, my intuition strengthened, enabling me to navigate new software environments effortlessly.

Uncovering the Hidden Threads:
Identifying similarities with applications I’d used before became my focus. I familiarized myself with common user interface patterns and leveraged transferable skills from past experiences, which significantly accelerated my learning. Embracing trial and error allowed me to learn from mistakes and refine my approach. By unveiling the hidden threads connecting interface patterns, transferable skills, and trial and error experiences, I quickly mastered new software.

Becoming an Analytical Visionary:
Applying analytical thinking, problem-solving skills, and visual thinking also played a crucial role in my software learning journey. I broke down complex tasks into manageable steps and analyzed the underlying logic of the software to comprehend its workings. Creating mental images or visual representations of information within the application aided in grasping complex concepts and systems. The combination of these three aspects empowered me to conquer challenges and navigate new software with ease.

Conclusion:
By applying the three “Software Whisperer” principles—Embracing the Intuitive Explorer, Uncovering the Hidden Threads, and Becoming an Analytical Visionary—I have significantly improved my ability to understand and use new software applications with little to no prior training. While patience and persistence are essential when venturing into unfamiliar digital terrain, the rewards are well worth it. I now conquer software challenges effortlessly, akin to a true Software Whisperer. I encourage you to try these principles for yourself and witness the difference they can make in your software learning journey. Trust me, they have been a game-changer for me, and I believe they can do the same for you too!

Time to come clean, I used ChatGPT to create this! I copied the original text (see link below) pasted into ChatGPT and experimented with different prompts.

Reference: https://www.linkedin.com/pulse/unlock-your-inner-software-whisperer-3-simple-vishvakanth-alexander%3FtrackingId=u0nGp05rTpawtiS5CF9NAQ%253D%253D/?trackingId=u0nGp05rTpawtiS5CF9NAQ%3D%3D

With a “Ranting Donald Trump” prompt:

Then, we’ve got the Hidden Threads. You won’t believe what I uncovered. I looked at other applications, and I saw similarities, big similarities! It’s like they were all connected, like a beautiful web, a tremendous web!

Posted on

Postman and Faker

I am a huge fan of Postman when developing API’s. I dont think i use even 20% of its abilities.

Today I found out that the Postman Pre-request scripts also have access to Faker.

Here is an example dynamic request body for creating a user.

const randomInt = Math.floor((Math.random()*100 +1));

const firstName = pm.variables.replaceIn("{{$randomFirstName}}");
const lastName = pm.variables.replaceIn("{{$randomLastName}}");
const domain = pm.variables.replaceIn("{{$randomCompanyName}}").replace(/[, ]+/g, ".").trim();


const body = {
firstName,
lastName,
email: ${firstName}.${lastName}@${domain}.com

}

pm.environment.set('body', JSON.stringify(body))

Here is Postman’s list of Faker options

https://learning.postman.com/docs/writing-scripts/script-references/variables-list/