Testing Extensions

Visual Studio Code supports running and debugging tests for your extension. These tests will run inside a special instance of VS Code named the Extension Development Host, and have full access to the VS Code API. We refer to these tests as integration tests, because they go beyond unit tests that can run without a VS Code instance. This documentation focuses on VS Code integration tests.

Overview

If you are migrating from vscode, see migrating from vscode.

If you are using the Yeoman Generator to scaffold an extension, integration tests are already created for you.

In the generated extension, you can use npm run test or yarn test to run the integration tests that:

  • Downloads and unzips latest version of VS Code.
  • Runs the Mocha tests specified by the extension test runner script.

Alternatively, you can find the configuration for this guide in the helloworld-test-sample. The rest of this document explains these files in the context of the sample:

The test script

VS Code provides two CLI parameters for running extension tests, --extensionDevelopmentPath and --extensionTestsPath.

For example:

  1. # - Launches VS Code Extension Host
  2. # - Loads the extension at <EXTENSION-ROOT-PATH>
  3. # - Executes the test runner script at <TEST-RUNNER-SCRIPT-PATH>
  4. code \
  5. --extensionDevelopmentPath=<EXTENSION-ROOT-PATH> \
  6. --extensionTestsPath=<TEST-RUNNER-SCRIPT-PATH>

The test script (src/test/runTest.ts) uses the vscode-test API to simplify the process of downloading, unzipping, and launching VS Code with extension test parameters:

  1. import * as path from 'path';
  3. import { runTests } from 'vscode-test';
  5. async function main() {
  6.   try {
  7.     // The folder containing the Extension Manifest package.json
  8.     // Passed to `--extensionDevelopmentPath`
  9.     const extensionDevelopmentPath = path.resolve(__dirname, '../../../');
  11.     // The path to the extension test runner script
  12.     // Passed to --extensionTestsPath
  13.     const extensionTestsPath = path.resolve(__dirname, './suite/index');
  15.     // Download VS Code, unzip it and run the integration test
  16.     await runTests({ extensionDevelopmentPath, extensionTestsPath });
  17.   } catch (err) {
  18.     console.error(err);
  19.     console.error('Failed to run tests');
  20.     process.exit(1);
  21.   }
  22. }
  24. main();

The vscode-test API also allows:

  • Launching VS Code with a specific workspace.
  • Downloading a different version of VS Code rather than the latest stable release.
  • Launching VS Code with additional CLI parameters.

You can find more API usage examples at microsoft/vscode-test.

The test runner script

When running the extension integration test, --extensionTestsPath points to the test runner script (src/test/suite/index.ts) that programmatically runs the test suite. Below is the test runner script of helloworld-test-sample that uses Mocha to run the test suite. You can use this as a starting point and customize your setup with Mocha’s API. You can also replace Mocha with any other test framework that can be run programmatically.

  1. import * as path from 'path';
  2. import * as Mocha from 'mocha';
  3. import * as glob from 'glob';
  5. export function run(): Promise<void> {
  6.   // Create the mocha test
  7.   const mocha = new Mocha({
  8.     ui: 'tdd',
  9.     color: true
  10.   });
  12.   const testsRoot = path.resolve(__dirname, '..');
  14.   return new Promise((c, e) => {
  15.     glob('**/**.test.js', { cwd: testsRoot }, (err, files) => {
  16.       if (err) {
  17.         return e(err);
  18.       }
  20.       // Add files to the test suite
  21.       files.forEach(f => mocha.addFile(path.resolve(testsRoot, f)));
  23.       try {
  24.         // Run the mocha test
  25.         mocha.run(failures => {
  26.           if (failures > 0) {
  27.             e(new Error(`${failures} tests failed.`));
  28.           } else {
  29.             c();
  30.           }
  31.         });
  32.       } catch (err) {
  33.         e(err);
  34.       }
  35.     });
  36.   });
  37. }

Both the test runner script and the *.test.js files have access to the VS Code API.

Here is a sample test (src/test/suite/extension.test.ts):

  1. import * as assert from 'assert';
  2. import { after } from 'mocha';
  4. // You can import and use all API from the 'vscode' module
  5. // as well as import your extension to test it
  6. import * as vscode from 'vscode';
  7. // import * as myExtension from '../extension';
  9. suite('Extension Test Suite', () => {
  10.   after(() => {
  11.     vscode.window.showInformationMessage('All tests done!');
  12.   });
  14.   test('Sample test', () => {
  15.     assert.equal(-1, [1, 2, 3].indexOf(5));
  16.     assert.equal(-1, [1, 2, 3].indexOf(0));
  17.   });
  18. });

Debugging the tests

Debugging the tests is similar to debugging the extension.

Here is a sample launch.json debugger configuration:

  1. {
  2.   "version": "0.2.0",
  3.   "configurations": [
  4.     {
  5.       "name": "Extension Tests",
  6.       "type": "extensionHost",
  7.       "request": "launch",
  8.       "runtimeExecutable": "${execPath}",
  9.       "args": [
  10.         "--extensionDevelopmentPath=${workspaceFolder}",
  11.         "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
  12.       ],
  13.       "outFiles": ["${workspaceFolder}/out/test/**/*.js"]
  14.     }
  15.   ]
  16. }

Tips

Using Insiders version for extension development

Because of VS Code’s limitation, if you are using VS Code stable release and try to run the integration test on CLI, it will throw an error:

  1. Running extension tests from the command line is currently only supported if no other instance of Code is running.

You can either use VS Code Insiders for development or launch the extension test from the debug launch config that bypasses this limitation.

Disabling other extensions while debugging

When you debug an extension test in VS Code, VS Code uses the globally installed instance of VS Code and will load all installed extensions. You can add --disable-extensions configuration to the launch.json or the launchArgs option of vscode-test‘s runTests API.

  1. {
  2.   "version": "0.2.0",
  3.   "configurations": [
  4.     {
  5.       "name": "Extension Tests",
  6.       "type": "extensionHost",
  7.       "request": "launch",
  8.       "runtimeExecutable": "${execPath}",
  9.       "args": [
  10.         "--disable-extensions",
  11.         "--extensionDevelopmentPath=${workspaceFolder}",
  12.         "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
  13.       ],
  14.       "outFiles": ["${workspaceFolder}/out/test/**/*.js"]
  15.     }
  16.   ]
  17. }
  1. await runTests({
  2.   extensionDevelopmentPath,
  3.   extensionTestsPath,
  4.   /**
  5.    * A list of launch arguments passed to VS Code executable, in addition to `--extensionDevelopmentPath`
  6.    * and `--extensionTestsPath` which are provided by `extensionDevelopmentPath` and `extensionTestsPath`
  7.    * options.
  8.    *
  9.    * If the first argument is a path to a file/folder/workspace, the launched VS Code instance
  10.    * will open it.
  11.    *
  12.    * See `code --help` for possible arguments.
  13.    */
  14.   launchArgs: ['--disable-extensions']
  15. });

Custom setup with vscode-test

Sometimes you might want to run custom setups, such as running code --install-extension to install another extension before starting your test. vscode-test has a more granular API to accommodate that case:

  1. import * as cp from 'child_process';
  2. import * as path from 'path';
  3. import {
  4.   downloadAndUnzipVSCode,
  5.   resolveCliPathFromVSCodeExecutablePath,
  6.   runTests
  7. } from 'vscode-test';
  9. async function main() {
  10.   try {
  11.     const extensionDevelopmentPath = path.resolve(__dirname, '../../../');
  12.     const extensionTestsPath = path.resolve(__dirname, './suite/index');
  13.     const vscodeExecutablePath = await downloadAndUnzipVSCode('1.40.1');
  14.     const cliPath = resolveCliPathFromVSCodeExecutablePath(vscodeExecutablePath);
  16.     // Use cp.spawn / cp.exec for custom setup
  17.     cp.spawnSync(cliPath, ['--install-extension', '<EXTENSION-ID-OR-PATH-TO-VSIX>'], {
  18.       encoding: 'utf-8',
  19.       stdio: 'inherit'
  20.     });
  22.     // Run the extension test
  23.     await runTests({
  24.       // Use the specified `code` executable
  25.       vscodeExecutablePath,
  26.       extensionDevelopmentPath,
  27.       extensionTestsPath
  28.     });
  29.   } catch (err) {
  30.     console.error('Failed to run tests');
  31.     process.exit(1);
  32.   }
  33. }
  35. main();

Migrating from vscode

The vscode module had been the default way of running extension integration tests and is being superseded by vscode-test. Here’s how you can migrate from it:

  • Remove vscode dependency.
  • Add vscode-test dependency.
  • As the old vscode module was also used for downloading VS Code type definition, you need to
    • Manually install @types/vscode that follows your engine.vscode in package.json. For example, if your engine.vscode is 1.30, install "@types/vscode": "^1.30.0".
    • Remove "postinstall": "node ./node_modules/vscode/bin/install" from package.json.
  • Add a test script. You can use runTest.ts in the sample as a starting point.
  • Point the test script in package.json to run the compiled output of runTest.ts.
  • Add a test runner script. You can use the sample test runner script as a starting point. Notice that vscode used to depend on mocha@4 and glob, and now you need to install them as part of your devDependencies.

Next steps