Automated Refactoring with JSCodeShift

How to Do Refactoring at Scale on a Large Codebase

Jan 21, 2024

Code refactoring is an important part of every software development project. The goal of code refactoring is usually to improve existing source code by migrating legacy code and reducing technical debt. It is therefore a very important software engineering practice to preserve the life of an application and to make it maintainable over time.

Refactoring can cover a wide range of topics. There are several examples, all of which have a more or less relative level of difficulty:

Replacing var by let and const for the declaration of variables in JavaScript
Migrating from one language to another: JavaScript to TypeScript
Migrating from one standard to another: proptypes to TypeScript's type aliases in React...

When you have a small project, you can dedicate time to do those migrations manually. But things start to be difficult when you have a large-scale project with thousands of files to migrate 😱 That's where tools like Facebook's jscodeshift come in. It can apply some changes to many files and therefore can be used to automate code refactoring. But before going into details about how that package works, let's first talk about a few things.

Abstract Syntax Tree (AST)

Abstract Syntax Trees are the output of one of the phases your code goes through when being compiled. When a source code is getting compiled, the compiler does the following:

Lexical Analysis (Tokenization): extracting all the tokens of your source code
- Keywords like let, const, while
- Identifiers like variables and functions names
- Literals like strings, numbers, variables values
- Operators, Separators, Comments, Punctuator...
Syntax Analysis (Parsing): to represent your source code in a way that helps to understand the structure of your source code. This phase takes as an input to all the tokens present in the code and builds a relation between them to check for example if you are declaring a variable or a function, if you are making an import statement, and so on. The output of this phase is the Abstract Syntax Tree (AST) which is represented as a tree data structure.
Code Generation: to generate the output of the compilation based on the information provided by the syntax analysis phase.

*Compiler Workflow Visualization by @_lrlna*

There are many parsers for the JavaScript language, almost all of them implementing the ESTree specification. The most famous libraries are Esprima by jQuery, Espree by eslint, acorn, and @babel/parser.

💡 Tip: you can use the amazing AST explorer as a playground to inspect the generated AST from a given JS source code.

Even if these techniques seem to be specific to compilation, they are used in many areas, such as in our code editors to do syntax highlighting based on tokens, by jest to generate snapshots and istanbul for code coverage calculation, for eslint for linting files...

The use case we are going to cover in this article is automating JavaScript refactoring with jscodeshift.

What Really is JSCodeShift?

JSCodeShift is an open-source library built around recast but providing a simpler API. It is a library that can:

Parse and generate Abstract Syntax Trees from a given JavaScript/TypeScript source code (using Esprima)
Traverse and manipulate the generated AST (adding, deleting nodes, renaming variables...) using ast-types, and then
Output the transformed AST code back to its base language (JavaScript/TypeScript).

JSCodeShift takes as an argument a transform file specifying the various changes to be made to the source code, and the list of files on which these changes must be applied. Example: jscodeshift -t transformer.ts src

Now enough theory. Let's create our first jscodeshift script together.

Our first example

For this example, let's go back to 2015 when ES6 was first introduced. One of the most important features back then was the ability to declare variables with other keywords than the traditional var: let to declare a changing variable and const to declare a constant.

The scenario is as follows: you have a project that was developed before 2015 and wanted to migrate to ES6 to modernize your source code. For that, you want to start first by replacing all occurrences of var present in the source code by let if the variable is reassigned somewhere in the code, or by const if it is not reassigned anywhere.

The first thing you need to do is to install jscodeshift on your machine: npm install -g jscodeshift

Then create and open two files: one file that will represent the source code and the other one that will be the transform file containing the jscodeshift script.

The process would be as follows:

Get all variable declarations var
For each variable declaration, check if the variable has changed since the declaration
- If the variable changed, replace var by let
- If not, replace var with const
Write back the modified source code

We first need to get all variable declarations.

Now for each variable declaration, we need to check if the variable has been updated. The ESTree specification defines two types of update operations:

Assignment Expression: when you reassign a value to an already declared variable
Update Expression: when you update the value of a variable (incrementing or decrementing)

So, to check if a variable value has changed in the file, we need to verify both operations. For that, we can create and use a helper to check if a variable changed, and use that helper to filter variable declarations that changed from the one that did not change.

Then we update the input source code by calling the rootSource.toSource() method.

To run the migration, we can run the command jscodeshift -t transformer.ts index.js

After running the script, the file content is updated 🤩

💡 Tip: If you wish so or if you do not have a development environment ready, you can play around with this example on AST explorer.

If you have to this far, congratulations! You can find more examples of jscodeshift transforms here.

Code BBBlog

Discussion about this post