Debugging using Delve DAP (aka dlv-dap)

The Go extension allows you to launch or attach to Go programs for debugging. You can inspect variables and stacks, setting breakpoints, and do other debugging activities using VS Code’s Debugging UI.

These debugging features are possible by using Delve, the Go debugger. The Go extension has been communicating with Delve through a custom debug adapter program. As the new Delve‘s native DAP implementation becomes available, the Go extension is now transitioning to skip the legacy debug adapter and directly communicate with Delve.

📣 We are happy to announce that now this new mode of Delve integration (dlv-dap mode) is ready for use in local debugging! We plan to make it default sometime in 2021 H2.

In this document, we assume you are using the new Delve’s native debug adapter implementation (dlv-dap).

Getting Started

To opt in to use this new adapter (dlv-dap), add the following in your VSCode settings.json.

    "go.delveConfig": {
        "debugAdapter": "dlv-dap",
    }

If you want to use dlv-dap for only a subset of your launch configurations, you can use the debugAdapter attribute to switch between “dlv-dap” and “legacy” mode. If you are using the Nightly version of this extension, dlv-dap is already the default debug adapter for local debugging scenarios, so the above setting is unnecessary.

Start Debugging

Open a file to debug (either package main source file or the test file) in the editor, and select the Run and Debug button from the Run view. Alternatively, you can start debugging using Start Debugging (F5) command from the Run menu or from the Command Palette (Linux/Windows: Ctrl+Shift+P, Mac: ⇧+⌘+P).

When no configuration is configured yet (no .vscode/launch.json file), the extension will choose a default configuration based on the file open in the editor.

If you already have launch configurations for the project (.vscode/launch.json), the Run view will display the configuration list to choose from.

❗ When you start debugging in dlv-dap mode for the first time, the extension will ask to install Delve built from head (dlv-dap). Please follow the instructions to install, and then start the debugging session again (i.e. selecting the source file, pressing F5 or click the codelens).

Watch “Go: Writing and debugging fast, reliable, and efficient software” to learn more about debugging features.

Please review the Features section that provides an overview of the debug UI and available features.

Staying Up-To-Date

Delve’s native DAP implementation is under active development, so take advantage of the most recent features and bug fixes by using Delve built from its master branch. The Go extension maintains this newest version of Delve separately from the officially released version of dlv and installs it with the name dlv-dap.

The easiest way to update dlv-dap on demand is to use the "Go: Install/Update Tools" command from the Command Palette (Linux/Windows: Ctrl+Shift+P, Mac: ⇧+⌘+P). The command will show dlv-dap in the tool list. Select dlv-dap, and the extension will build the tool at master.

Once dlv-dap is installed on your system, the extension will prompt you for update whenever installing a newer version is necessary (usually after the Go extension upgrade). You can set the go.toolsManagement.autoUpdate setting so the extension can update dlv-dap automatically for you.

If you need to install dlv-dap manually outside of VS Code (for example, you are building a dev container with necessary tools preinstalled), please see the Manual Installation section.

Features

For general debugging features such as inspecting variables, setting breakpoints, and other activities that aren’t language-dependent, review VS Code debugging.

Configure

When you need more than the default debugging setup, you can create a launch configuration file for the project by clicking the “create a launch.json file” link in the Run view. Then, choose from the debug configuration drop-down menu. VS Code will create a launch.json file in a .vscode folder in your workspace (project root folder) or in your user settings or workspace settings.

If you already have a launch.json for your project, you can open it using Open launch.json.

To add a new configuration to an existing launch.json, choose the “Add Configuration…” button to invoke the snippet IntelliSense.

There are many configuration attributes (see the Launch.json attributes section). IntelliSense in VS Code’s launch.json editor will help you navigate available options and documentation.

Launch

You can choose “Start Debugging (F5)” and “Run Without Debugging (^F5)” a.k.a the noDebug mode. This feature uses a launch request type configuration. Its program attribute needs to be either the go file or folder of the main package or test file. In this mode, the Go extension will start the debug session by building and launching the program. The launched program will be terminated when the debug session ends.

• Supported modes
  • debug: build and debug a main package
  • test: build and debug a test
  • exec: debug a precompiled binary. The binary needs to be built with -gcflags=all=”-N -l” flags to avoid stripping debugging information.
  • auto: automatically choose between debug and test depending on the open file.

Attach

You can debug an already running program using the attach request type configuration. With the attach request, the Go extension starts dlv-dap and configures it to attach to the specified process. Users can select the process to debug with one of the following options:

• Specifying the numeric process id (PID) with the processId attribute.
• Specifying the target program name in the processId attribute. If there are multiple processes matching the specified program name, the extension will show the list of matching processes at the start of the debug session.
• Specifying 0 in the processId attribute and selecting the process from the drop-down menu at the start of the debug session.

When you end the debug session, the debug UI allows you to choose to either

• Disconnect: detach and leave the process running. (default)
• Terminate: terminate the attached process.

Debug Actions

Once a debug session starts, the Debug toolbar will appear on the top of the editor.

• Continue / Pause F5
• Step Over (aka next in Delve) F10
• Step Into (aka step in Delve) F11
• Step Out (aka stepout in Delve) Shift+F11 or ⇧F11
• Restart (currently this is “Stop + Start”) Ctrl+Shift+F5 or ⇧⌘F5
• Stop (terminate the debugee. Available in Launch request) Shift+F5 or ⇧F5
• Disconnect (detach from the debugee. Available only in Attach request) Shift+F5 or ⇧F5
• Terminate (terminate the debugee. Available only in Attach request) Alt+Shift+F5 or ⌥⇧F5

Breakpoints

See VS Code’s Debug Documentation on Breakpoints to get familiar with VS Code’s UI. The Go debugger supports multiple ways to configure breakpoints.

• Breakpoints: you can set breakpoints by clicking on the editor margin or using F9 on the current line. If the breakpoints can’t be set by Delve, VS Code will show the failure reason and grey out the dot.

• Conditional breakpoints: you can specify breakpoint conditions (similar to Delve’s condition command).
  • Expression condition: takes a boolean expression.
  • Hit count: supports comparison operators (>, >=, <, <=, ==, !=) with an integer value. % n form means we should stop at the breakpoint when the hitcount is a multiple of n.

• Function Breakpoints: breakpoints can be set based on function names. Press the + button in the BREAKPOINTS section header and enter the location in the form of <function>[:<line>]. This sets the breakpoint in the line inside the function. The full syntax for function is <package>.(*<receiver type>).<function_name> as specified in Delve’s location spec. Function breakpoints are shown with a red triangle in the BREAKPOINTS section.

• Logpoint (WIP)

Data Inspection

You can inspect variables in the VARIABLES section of the Run view or by hovering over their source in the editor. Variable values and expression evaluation are relative to the selected stack frame in the CALL section.

By default, the VARIABLES section hides global variables, and shows only local variables and function arguments. However, you can still inspect global variables from the DEBUG CONSOLE panel. If you prefer to have the VARIABLES section show global variables, set the showGlobalVariables attribute in the launch.json configuration, or set it in the go.delveConfig setting.

When you select a variable and right click from the VARIABLES section, the context menu will present shortcuts to features such as:

• Set Value: you can set/modify simple string, numeric, pointer values. Using composite literals, or memory allocation is not supported.
• Copy Value: this copies the value in clipboard.
• Copy as Expression: this is useful when you need to query from the REPL in the DEBUG CONSOLE panel.
• Add to Watch: this will automatically add the expression to the WATCH section.

Shadowed variables will be marked with ().

⚠️ Delve debugger imposes variable loading limits to prevent loading too many variables at once and negatively impacting debugging latency. The dlv-dap mode uses a different approach. It takes advantage of the interactive UI features to provide on-demand loading of individual variables, paging of arrays, slices and maps and increased string limits depending on the context. We continue to explore additional interactive features to balance performance and usability of variable loading and look forward to your feedback.

You can inspect variables and evaluate expressions from the DEBUG CONSOLE panel too. Acceptable expressions are either

• A valid Delve expression, or
• call <function_call_expression> to call functions.

Variables and expressions accepted in DEBUG CONSOLE can be also registered in the Run view’s WATCH section, so they can be evaluated automatically as you debug. The “Add to Watch” feature from the VARIABLES section is convenient when you want to register interesting variables.

⚠️ Function call feature is highly EXPERIMENTAL due to the limitation in Go runtime. Registering function calls in the WATCH section can often be problematic. Pause, stop, and disconnect will not work while a function call is running.

Hover over variables in editors during debugging shows the value of the variable. For this feature, VS Code extracts the variable expression and makes a request to the debugger to evaluate the expression. Delve evaluates the expression relative to the highlighted stack frame chosen in the CALL STACK. By default, that is the current top-most frame.

⚠️ Limitation

• VS Code heuristically determines the variable expression without full understanding of the scope & the currently selected frame. Delve tries to evaluate the provided expression in the selected frame. As a result, hover over variables outside the selected frame’s function may present incorrect information.

Call Stack

You can inspect all goroutines and their stacks in the CALL STACK section. The CALL STACK section UI allows switching between goroutines or selecting a different stack frame. As a different stack frame or different goroutine is selected, the scope shown in the VARIABLE section will be updated for the newly selected stack frame, and the expressions in the WATCH section will be automatically reevaluated relative to the newly selected stack frame.

1. Goroutine stacks are annotated with their internal goroutine IDs.
2. The current goroutine is marked with *. If multiple goroutines stop (e.g. hit breakpoints) concurrently, Delve will pick one randomly. There also might not be a current goroutine (e.g. deadlock, pause or internal breakpoint hit by a system thread not running a goroutine).
3. If you click a goroutine call stack from the CALL STACK section, the goroutine is selected.
4. You can select a frame of the selected goroutine. The VARIABLE and WATCH sections will be updated accordingly and the cursor in the editor will be moved to the corresponding location in the source code.
5. Runtime stack frames are deemphasized (greyed out or collapsed).
6. Thread IDs are shown for scheduled goroutines.
7. Stop reason. It’s possible that there are multiple reasons goroutines were stopped, but currently only one reason is presented.
8. File name and line number of the frame.
9. You can trigger a debug action with the selected goroutine. Note: Resuming or stopping only a single goroutine (Go Issue 25578, 31132) is currently not supported, so the action will cause all the goroutines to get activated or paused.
10. Function name of the frame.

When the program stops due to exception, panic, or bad access error, the CALL STACK shows the stop reason and the editor highlights the source location with more details.

Configuration

Launch.json Attributes

There are many attributes that you can adjust in the launch and attach debug configuration. The following general attributes are mandatory for all launch configurations.

• name: the name of your configuration as it appears in the drop-down in the Run view.
• type: the debugging type VS Code uses to decide which debugging extension should be used. Always leave this set to ”go”.
• request: launch or attach.

Here is the list of attributes specific to Go debugging.

⚠️ dlv-dap needs file or directory values in the launch configuration to be absolute paths. When configuring those values, use the VS Code variables substitution - VS Code will resolve the variables inside strings in launch.json before passing the configuration to the Go extension and dlv-dap. For example, ${workspaceFolder} will be replaced with the absolute path to the workspace root folder. When appropriate, the Go extension will resolve relative paths or home directory (~) before sending the configuration to dlv-dap.

Debugging symlink directories

Since the debugger and go compiler use the actual filenames, extra configuration is required to debug symlinked directories. Use the substitutePath property to tell the debugAdapter how to properly translate the paths. For example, if your project lives in /path/to/actual/helloWorld, but the project is open in vscode under the linked folder /path/to/hello, you can add the following to your config to set breakpoints in the files in /path/to/hello:

{
    "name": "Launch remote",
    "type": "go",
    "request": "launch",
    "mode": "debug",
    "program": "/path/to/actual/helloWorld",
    "substitutePath": [
        {
            "from": "/path/to/hello",
            "to": "/path/to/actual/helloWorld",
        },
    ],
}

Settings

You can adjust the default value of the following configuration properties using go.delveConfig settings. These default values are useful when you choose to run a debug session without the launch configuration set in launch.json. For example, debug sessions started using the Debug Test code lenses use the adjusted values from these settings.

• go.delveConfig
  • debugAdapter: Controls which debug adapter to use (default: legacy). Select ‘dlv-dap’.
  • showGlobalVariables: Show global variables in the Debug view (default: false).
  • substitutePath: Path mappings to apply to get from a path in the editor to a path in the compiled program (default: []).

⚠️ Where is the dlvLoadConfig setting? Delve debugger imposes variable loading limits to avoid loading too many variables at once and negatively impacting debugging latency. The legacy adapter supported dlvLoadConfig to adjust these limits for the duration of the session. The user therefore had to come up with a one-size-fits-all limit if the default behavior was not satisfactory. dlv-dap mode uses a different approach as described in the Data Inspection section. If this setting is configured and dlv-dap mode is used, the extension will show a warning prompt now. If the current variable loading behavior and internal limits are not working for you, please open an issue and share your feedback.

Advanced Topics

Go Debug Extension Architecture Overview

VS Code implements a generic, language-agnostic debugger UI based on Debug Adapter Protocol (DAP), an abstract protocol for communicating with debugger backend. Previously, the Go extension used an intermediary typescript program (legacy debug adapter) to launch Delve and adapt Delve to DAP. With the new, native DAP implementation in Delve, the intermediary program is no longer necessary, and efficient and tight integration with Delve becomes possible.

For information on debugging using the legacy debug adapter, please see the old Debugging Documentation. Note that many new or enhanced features discussed in this document may not be available with the legacy debug adapter.

Manually installing dlv-dap

On rare occasions, you may want to install dlv-dap by yourself instead of letting the extension handle its installation.

First, find where the Go extension finds tools. Like other tools the extension uses, the Go extension searches the dlv-dap executable from ${GOPATH}/bin, ${GOBIN} and ${PATH} (or Path in Windows). So, install dlv-dap in the directory. The easiest way to check the tool installation location the Go extension uses is currently by running the Go: Locate Configured Go Tools command from the command palette (⇧+⌘+P or Ctrl+Shift+P).

The following commands download the source of Delve from the master branch, build & store as dlv-dap in ~/go/bin/ directory assuming the directory is the place you found from the first step.

If your Go version is 1.16 or newer:

$ GOBIN=/tmp/ go install github.com/go-delve/delve/cmd/dlv@master
$ mv /tmp/dlv $GOPATH/bin/dlv-dap

If your Go version is older than 1.16:

$ cd $(mktemp -d)
$ GO111MODULE=on GOBIN=/tmp/ go get github.com/go-delve/delve/cmd/dlv@master
$ mv /tmp/dlv $GOPATH/bin/dlv-dap

If you want to explicitly specify the location of the delve binary, use the go.alternateTools setting:

"go.alternateTools": {
    "dlv-dap": "<absolute path to your dlv binary>"
}

Remote Debugging

If you are able to use the Remote Development extensions and VS Code’s universal remote development capabilities, that is the recommended way to remote debug Go programs. Check out the Getting started section and Remote tutorials to learn more.

Remote debugging is the debug mode where the debug target runs in a different machine or a container. Support for remote debugging using Delve’s native DAP implementation is still a work-in-progress. This section describes a current temporary workaround and its limitations. If the following workaround is not working for your case, please file an issue and help us understand remote debugging use cases better.

When using the dlv-dap mode, the delve instance running remotely needs to be able to process DAP, instead of the traditional Delve JSON-RPC. The following command starts a Delve DAP server on port 12345 and outputs dap specific events.

$ dlv-dap dap --listen=:12345

Then, use the following launch configuration:

{
  “name”: “Connect to server (DAP)”,
  “type”: “go”,
  “request”: “launch”,
  “port”: 12345,
  “host”: “127.0.0.1”,
  “mode”: “exec”,
  “program”: “/path/to/remote/workspace/program/executable”,
  “substitutePath”: [
      { “from”: ${workspaceFolder}, “to”: “/path/to/remote/workspace” },
      ...
  ]
}

When seeing the ”port” attribute being used in the launch request, Go extension will assume a Delve DAP server is started externally and accessible through the specified host:port and tell VS Code to connect to it directly. The program attribute must point to the absolute path to the package or binary to debug in the remote host’s file system even when substitutePath is specified.

⚠️ Limitations

• Unlike dlv <debug|exec|attach> --headless commands traditionally used for remote debugging scenarios, Delve’s new dap sub command does not launch or attach to the debuggee process until it receives a Launch/Attach request. We understand this limitation, and we are currently working on addressing this limitation.
• Anyone who can connect to the Delve DAP server’s host:port can exploit it to run arbitrary programs.
• When using ”attach” requests, you will need to specify the processId since the processId resolution feature cannot gather process information running remotely.
• Delve DAP does not support --allow-multiclient or --continue flags yet, which means after a debug session ends, the dlv-dap process will exit.
• If you use debug or test mode launch requests, Delve builds the target binary. Delve tries to build the target from the directory where the dlv (or dlv-dap) process is running, so make sure to run the dlv-dap command from the directory you’d run the go build or go test command.

Running Debugee Externally

Sometimes you’d like to launch the program for debugging outside VS Code (e.g., as a workaround of the missing console support), there are currently two options.

• Compile and run the program from the external terminal and use the “attach” configuration.
• Use “Remote Debugging”; run dlv-dap dap --listen=:<port> from the external terminal, and set the ”port” attribute in your launch configuration.

Reporting issues

The VS Code Go maintainers are reachable via the issue tracker and the #vscode channel in the Gophers Slack.

Please reach out on Slack with questions, suggestions, or ideas. If you have trouble getting started on an issue, we’d be happy to provide pointers and advice.

When you are having issues in dlv-dap mode, first check if the problems are reproducible after updating dlv-dap. It’s possible that the problems are already fixed. Follow the instruction for updating dlv-dap) and updating extension.

Please report issues in our issue tracker with the following information.

• go version
• go version -m <path/to/dlv-dap>
• VS Code and VS Code Go version.
• Instructions to reproduce the issue (code snippets, your launch.json, screenshot)

Developing

Code location

The core part of Delve DAP implementation is in the service/dap package. Follow Delve project’s contribution guideline to send PRs.

Code for integration with the Go extension is mostly in src/goDebugFactory.ts and tests are in test/integration/goDebug.test.ts. Please take a look at VS Code Go project’s contribution guideline to learn about how to prepare a change and send it for review.

Testing

For simple launch cases, build the delve binary, and configure "go.alternateTools" setting.

"go.alternateTools": {
    "dlv-dap": <path_to_your_delve>
}

Set logOutput and showLog attributes in launch.json to enable Delve-side logging (server-side) and DAP message tracing.

{
    "name": "Launch file",
    "type": "go",
    "debugAdapter": "dlv-dap",
    "showLog": true,
    "logOutput": "dap",
    ...
}

Set trace attribute to control the verbosity of Go extension’s side logging (client-side).

{
    "name": "Launch file",
    "type": "go",
    "debugAdapter": "dlv-dap",
    "trace": "verbose",
    ...
}

The logging will appear in the Go Debug output channel (Command Palette -> “View: Toggle Output” -> Select “Go Debug” from the dropdown menu).

$ dlv-dap dap --listen=:12345 --log --log-output=dap

{
    "name": "Launch file",
    "type": "go",
    "request": "launch",
    "debugAdapter": "dlv-dap",
    ...
    "port": 12345
}