Add go-to-definition for symbols in doc comments, Markdown and Tutorial files by Padmashree06 · Pull Request #2698 · swiftlang/sourcekit-lsp

Padmashree06 · 2026-06-20T17:01:34Z

This PR introduces go-to-definition support for Markdown and Tutorial files along with symbols referenced in documentation comments in Swift files.

To enable this functionality, symbol graphs are now emitted during both the initial build and background indexing. The generated symbol graph data is then used to resolve symbols referenced in documentation files and navigate to their corresponding source definitions.

Changes

Emit symbol graphs during the initial build.
Emit symbol graphs during background indexing.
Add go-to-definition support for Markdown and Tutorial files using the generated symbol graph data.
Add go-to-definition support for symbols referenced in documentation comments in Swift files.

User Impact

Users can now invoke go-to-definition on symbols referenced in Markdown and Tutorial files, as well as Swift documentation comments, and navigate to the corresponding source definition.

matthewbastien · 2026-06-26T17:40:52Z

@swift-ci please test

ahoppen

Sorry for taking so long to review this. I left some thoughts inline. Also: Could you add some test cases that test the new behavior?

ahoppen · 2026-07-07T10:03:39Z

      arguments += ["--traits", traits.joined(separator: ",")]
    }
    arguments += toolsets.flatMap { ["--toolset", $0.pathString] }
-    arguments += pkgConfigDirectories.flatMap { ["--pkg-config-path", $0.pathString] }


Unintentionally removed?

Yes, that was unintentional. I'll add it back!

ahoppen · 2026-07-07T10:04:47Z

+      "-Xswiftc", "-Xfrontend", "-Xswiftc", "-emit-symbol-graph-dir",
+      "-Xswiftc", "-Xfrontend", "-Xswiftc", symbolGraphPath,
+    ]
+


We shouldn’t emit symbol graphs during target preparation. The goal of target preparation is to get .swiftmodule files out there as quickly as possible so that we can index the files depending on those modules. So, all the symbol graph generation should happen during indexing, ie. UpdateIndexStoreTaskDescription

ahoppen · 2026-07-07T10:05:02Z

+  }
+
+  package func definition(_ req: DefinitionRequest) async throws -> LocationsOrLocationLinksResponse? {
+


Superfluous newline

ahoppen · 2026-07-07T10:05:21Z

+  package func symbolInfo(_ req: SymbolInfoRequest) async throws -> [SymbolDetails] {
+    return []


Do we need this stub?

Yes, this stub is required. Without it, DocumentationLanguageService returns a "request not implemented" error when SymbolInfo is called through IndexBasedDefinition. As a result, IndexBasedDefinition never returns an empty array, so the fallback condition if indexBasedResponse.isEmpty { ... } is never reached.

With this stub, IndexBasedDefinition returns an empty array, allowing the fallback to execute and enables go-to-definition for DocC comments, tutorials, and Markdown files.

Maybe we should do the fallback path even if indexBasedDefinition throws then, similar to how we try more language services if one of them throws. I’d just like to log all the errors thrown by both indexBasedDefinition and all language services tried in case definition fails.

ahoppen · 2026-07-07T10:07:02Z

+    let converter = SourceLocationConverter(fileName: "", tree: sourceFile)
+    let cursorLine = converter.location(for: absolutePosition).line
+    let cursorColumn = converter.location(for: absolutePosition).column


Instead of creating a SourceLocationConverter, there should be some method on DocumentSnapshot that gives you the source location you need. That’s going to be faster because it already has a pre-populated line table.

ahoppen · 2026-07-07T10:13:53Z

+    if snapshot.language == .swift {
+      clickedSymbol = extractSymbolFromDocComment(snapshot: snapshot, at: req.position)
+    } else {
+      clickedSymbol = extractSymbolFromText(snapshot.text, at: req.position)


Should this only be done for Markdown files? I assume you don’t want to parse eg. C files as Markdown.

The else block is only reached for Markdown and Tutorial files. Since DocumentationLanguageService is registered only for .swift, .markdown, and .tutorial files, it won't attempt to parse C files as Markdown.

I would still add the check. It’s not unreasonable to assume that at some point we’ll also want to support C-based languages for Docc comments.

Yes! Will add it!

ahoppen · 2026-07-07T10:15:45Z

+    guard let data = try? Data(contentsOf: targetGraphURL),
+      let jsonObject = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
+      let symbolsArray = jsonObject["symbols"] as? [[String: Any]]


Please use a Codable struct here instead of relying on JSONSerialization, which really dates back to Objective-C.

ahoppen · 2026-07-07T10:16:15Z

+      workspaceRoot
+      .appendingPathComponent(".build/symbol-graphs")
+      .appendingPathComponent("\(moduleName).symbols.json")


Does this also work if the referenced symbol is in a different module than the current file?

No, currently it does not.

ahoppen · 2026-07-07T10:18:30Z

+
+private let updateSymbolGraphIDForLogging = Atomic<UInt32>(1)
+
+package struct UpdateSymbolGraphTaskDescription: IndexTaskDescription {


Would it be possible to generate the index store during the current indexing operation UpdateIndexStoreTaskDescription? Both indexing and symbol graph generation need to parse and type-check the file and we should be able to share that work.

I tried combining them initially, but the symbol graph wasn't getting generated. From what I found, symbol graph generation requires a module-level build, while the index store task does a file-level build. So, to keep the module-level compilation cheap, UpdateSymbolGraphTask uses the -experimental-skip-all-function-bodies flag, but using that flag for the index store task wouldn't work well, since we'd need to parse and type-check the code inside function bodies too.
So I kept them separate for now, but if you see a way to combine them, I'd be happy to give it a try.

From what I found, symbol graph generation requires a module-level build

I can’t think of why that would be the case. During debug builds, the driver will already invoke multiple swift-frontend commands with only a subset of the files in them, so we should be able to generate a symbol graph for individual files as well. It could be that symbol graph generation is incompatible with some of the flags we pass for indexing, but I’d like to understand what they are before making any decisions here.

Yes, Symbol Graph Generation is incompatible with -index-file, -index-store-path and -disable-batch-mode flags.

I just dug into the compiler and symbol graph extraction happens while writing out the module file here. Technically, I think all the information for symbol graph generation should be available after type checking, there’s no need to also run SILGen to write it out. That also explains why symbol graph extraction is incompatible with -index-file, which stops after type checking and doesn’t emit a Swift module.

Ideally, we should investigate whether we could pull the symbol graph generation to an earlier place in the compilation pipeline so that it can run together with -index-file. But I suppose that’s out of scope for your GSoC project.

With that in mind, I think we might be fine here because we use -experimental-skip-all-function-bodies, which should bound the time spent during symbol graph generation. Still, I would like to see measurements how how long symbol graph generation takes compared to indexing when running background indexing on a reasonably sized project like SourceKit-LSP, SwiftPM, SwiftSyntax or whatever your favorite is. Should symbol graph generation take more time than indexing, I would be extremely hesitant to take this change outside of an experimental feature flag, if it adds <20%, I would probably be fine, anything in between we’d need to weigh our options.

I’m also going to leave a few comments on the compiler argument processing that you’re doing, I think there’s some room for improvement.

Also, I’d like to hear @hamishknight and @rintaro’s opinions.

These trials were performed on SourceKit-LSP

Trial Time taken for SymbolGraph generation (ms) Time taken for Indexing (ms) Overhead percentage (%) Performance cost

1 159,494 63,872.3 249.70 2.5×

2 171,342 82,867.8 206.76 2×

3 352,348 146,415 240.65 2.4×

Average overhead percentage = 232.37%
Average Performance cost = 2.3x

ahoppen · 2026-07-07T10:20:53Z

+          }
+        }
+        if isInDocComment {
+          throw ResponseError.requestNotImplemented(DefinitionRequest.self)


I feel like requestNotImplemented is the wrong signal here because the method is indeed implemented. I’d rather return nil to indicate that the language service wasn’t able to resolve a symbol and then SourceKitLSPServer can try the next language service.

From my understanding returning nil won't fall back to the next registered language service. SourceKitLSPServer.handleRequest only continues to the next language service when it catches a ResponseError with the .requestNotImplemented code. A plain nil return is treated as a successful result, so the loop stops there, because of which the DocumenationLanguageService is never triggered for Swift files and hence the go-to-definition feature for DocC comments in Swift files will not work.

If I'm missing something or there's a better way to approach this, please let me know.

From my understanding returning nil won't fall back to the next registered language

I don’t think there’s a reason why we can’t change that behavior.

Yes, changed the behavior to continue to next language service on returning nil.

matthewbastien · 2026-07-09T13:14:12Z

@swift-ci please test

ahoppen

The more I think about this, the more I come to the conclusion that to support background symbol graph generation, we’d need to re-work how all of this is done in the compiler.

Symbol graph generation should be possible after type checking, same as background indexing and not rely on module generation
Symbol graph generation should be possible on a per-file basis instead of requiring the entire module to be compiled
And once the two items above are done, it’s likely easy to make it compatible with -index-file and share one compiler invocation.

But this is likely going to be a huge endeavor on its own and I have no idea what the scope is, or if it’s even possible. Sorry that I can’t be more helpful here.

Maybe @hamishknight and @rintaro have thoughts.

ahoppen · 2026-07-17T09:24:53Z

+      while i < rawArgs.count {
+        let arg = rawArgs[i]
+
+        if arg == "-I" || arg == "-F" || arg == "-target" || arg == "-sdk"


Why are all of these transformations necessary when they aren’t necessary for background indexing? Generally, indexing only works for the contents of the files on disk, so we shouldn’t have any vscode-local: files.

Thank you for pointing it out! Because fallbackAfterTimeout was true, when the build server didn't respond in time, it would fall back to locally-synthesized settings that could still carry vscode-local:-scheme paths, because of which the transformation was necessary. Now that fallbackAfterTimeout is set to false this transformation is not necessary. I've removed it!

ahoppen · 2026-07-17T09:25:29Z

+      if !baseArgs.contains("-wmo") && !baseArgs.contains("-whole-module-optimization") {
+        baseArgs.append("-wmo")


Why do we need -wmo? Symbol graph generations should work without any kind of optimization, I would imagine.

Yes, symbol graph generation works without -wmo flag, but the generated symbol graph will be incorrect as the cross-module symbol references (e.g. String, Int, Void, Escapable/Copyable) resolve to broken/incomplete precise identifiers .-wmo type-checks the entire module in a shared context rather than processing each file independently, which is necessary for these cross-file references to be resolved correctly.

ahoppen · 2026-07-17T09:25:44Z

+      var cleanEnvironment = ProcessInfo.processInfo.environment
+      cleanEnvironment.removeValue(forKey: "SWIFT_DRIVER_SUPPLEMENTARY_OUTPUT_FILE_MAP")
+      cleanEnvironment.removeValue(forKey: "SWIFT_DRIVER_OUTPUT_FILE_MAP")
+      cleanEnvironment.removeValue(forKey: "SWIFT_DRIVER_TEMP_DIR")
+      cleanEnvironment.removeValue(forKey: "SWIFT_DRIVER_RESPONSE_FILE_PATH")
+      cleanEnvironment.removeValue(forKey: "SWIFT_DRIVER_TOOLNAME")


Why are these needed if they aren’t needed for background indexing?

There was previously a conflict between the SWIFT_DRIVER_* environment variables and the explicit compiler flags this task passes, for instance, SWIFT_DRIVER_SUPPLEMENTARY_OUTPUT_FILE_MAP conflicting with -emit-module-path, since both specified the module output location. These conflicts no longer occur, so I've removed the environment cleanup that was working around it.

ahoppen · 2026-07-17T09:27:38Z

+      let swiftc: URL
+
+      do {
+        guard let firstFile = filesToIndex.first?.file.mainFile else { return }


We shouldn’t assume that all files in the batch have the same build settings. See what UpdateIndexStoreTaskDescription does to collect files that have the same build settings.

ahoppen · 2026-07-17T09:34:42Z

+      )
+
+      // emit module and symbol graph
+      let args =


If I understand correctly, we are always emitting a symbol graph for the entire module here. This means that if indexing batches a module with files a.swift, b.swift, c.swift as [a.swift, b.swift], [c.swift], we would be invoking symbol graph generation for the same module twice. What’s worse is that these can run concurrently and potentially interfere with their writes to disk.

Furthermore, this means that an edit to a file in a module with, say, 100 files, means that we need to re-generate the symbol graph for the entire module, which could take a while (happy to be proven otherwise based on some real-world projects). During background indexing, we have -index-file-path to only index a subset of the files listed in the compiler invocation. Really, symbol graph generations should have the same.

The best solution I can think of right now is to make UpdateSymbolGraphTaskDescription operate on targets instead of source files (because it generates a symbol graph for all files in the target at once), but then that’s going to be incompatible with symbol graph generation for clang-based files, for which we can generate a symbol graph on a per-file basis.

But overall, this whole-module approach is giving me extremely uneasy feelings about this entire endeavor. Files get saved very frequently and having to re-generate a .swiftmodule for the file’s entire module is going to be a very significant cost.

ahoppen · 2026-07-17T09:39:00Z

+          "-emit-module-path", moduleOutputPath,
+          "-Xfrontend", "-emit-symbol-graph",
+          "-Xfrontend", "-emit-symbol-graph-dir", "-Xfrontend", symbolGraphDir.pathString,
+          "-Xfrontend", "-experimental-skip-all-function-bodies",


Does this succeed in case a file in the current module contains a compilation error? If so, you could try adding -experimental-allow-module-with-compiler-errors.

Yes,I've added the flag

ahoppen · 2026-07-17T09:39:49Z

+          "-emit-module-path", moduleOutputPath,
+          "-Xfrontend", "-emit-symbol-graph",
+          "-Xfrontend", "-emit-symbol-graph-dir", "-Xfrontend", symbolGraphDir.pathString,
+          "-Xfrontend", "-experimental-skip-all-function-bodies",


Other flags that may be worth investigating are: -experimental-lazy-typecheck, -experimental-skip-non-exportable-decls. We set those during background preparation and they may apply here as well.

ahoppen · 2026-07-17T09:41:00Z

+  }
+
+  package func execute() async {
+    await withLoggingSubsystemAndScope(subsystem: indexLoggingSubsystem, scope: "update-symbolgraph-\(id % 100)") {


Similar to UpdateIndexStoreTaskDescription’s indexStoreUpToDateTracker, we should have something equivalent here.

Yes I have added symbolGraphUpToDateTracker!

ahoppen · 2026-07-17T09:43:35Z

+
+  package func execute() async {
+    await withLoggingSubsystemAndScope(subsystem: indexLoggingSubsystem, scope: "update-symbolgraph-\(id % 100)") {
+      guard language == .swift else { return }


Do you have plans to support c-based languages as well?

Only Swift documentation is planned for this GSoC project right now.

ahoppen · 2026-07-17T09:48:55Z

+      let args =
+        [swiftc.path] + baseArgs + [
+          "-emit-module",
+          "-emit-module-path", moduleOutputPath,


Do we ever need these .swiftmodules? If not, we should be able to delete them straight away (or stream them to /dev/null, but really, we shouldn’t be generating them at all in the compiler.

We would require updated .swiftmodules as Symbol graph generation relies on them. Only if the .swiftmodules are updated the symbol-gaphs will be updated.

snprajwal · 2026-07-18T13:49:04Z

The more I think about this, the more I come to the conclusion that to support background symbol graph generation, we’d need to re-work how all of this is done in the compiler.

Symbol graph generation should be possible after type checking, same as background indexing and not rely on module generation

The symbol graph extractor is designed to work with a ModuleDecl input from any source, so it's reasonable to wire up a path that allows symbol graph generation after typechecking. It's mostly straightforward to do AFAIU. As mentioned above, this may potentially be out of scope for this project.

Symbol graph generation should be possible on a per-file basis instead of requiring the entire module to be compiled

And once the two items above are done, it’s likely easy to make it compatible with -index-file and share one compiler invocation.

This is significantly harder, and I'm not sure if it's feasible. There are cross-file relationships that depend on the full module being compiled:

Protocol conformances
Extensions for types in other files
Synthesised symbols
Default implementation members

If the generation is done on a per-file basis, the information available would be incomplete, and the resulting symbol graph would also diverge from the contents of the module-level symbol graph. Since the relationships are cross-file, editing a file would require re-processing the file's dependents to correctly update the state of the graph.

snprajwal · 2026-07-18T18:26:36Z

I looked into SymbolGraphGen and added support for emitting symbol graphs with -typecheck here: swiftlang/swift#90799

Some numbers comparing this with the existing symbol graph generation after serialisation (ran with hyperfine, mean of 20 runs with -Onone):

Project	Module	Files	Serialised	Typecheck	Speedup
sourcekit-lsp	SourceKitLSP	31	1.831 ± 0.037 s	0.837 ± 0.027 s	2.19 ± 0.08x
swift-syntax	SwiftParser	46	3.517 ± 0.025 s	1.557 ± 0.022 s	2.26 ± 0.04x
swift-package-manager	PackageModel	59	2.850 ± 0.029 s	1.188 ± 0.054 s	2.40 ± 0.11x
swift-build	SWBUtil	100	5.259 ± 0.092 s	2.206 ± 0.068 s	2.38 ± 0.08x

This should help here in terms of the performance overhead of using symbol graphs. It does not alleviate our concerns about needing to typecheck the whole module instead of a single file, but it can make it somewhat less burdensome.

…generation

Padmashree06 added 6 commits June 8, 2026 13:50

Add compiler flag to emit symbol graph during background indexing

7deb27a

Emit symbol graph during initial build

a55ed3d

emit dynamic modules for symbol graph

ace236b

combined emit module and symbol graph generation into a single command

6fb763f

Emit symbol graph during background indexing

bc1f9c3

Add go-to-definition for .md and .tutorial files

501d5ac

Padmashree06 requested review from ahoppen, hamishknight and rintaro as code owners June 20, 2026 17:01

Add go-to-definition for doccComments in swift files

9434fe4

Padmashree06 changed the title ~~Emit Symbol Graphs and add go-to-definition support for Markdown and Tutorial files~~ Add go-to-definition for symbols in doccComments, Markdown and Tutorial files Jun 23, 2026

Padmashree06 changed the title ~~Add go-to-definition for symbols in doccComments, Markdown and Tutorial files~~ Add go-to-definition for symbols in doc comments, Markdown and Tutorial files Jun 23, 2026

Padmashree06 added 3 commits June 30, 2026 11:15

Merge branch 'main' into emiting-symbol-graph

9655a1e

Merge branch 'swiftlang:main' into emiting-symbol-graph

789e95a

Fix optional unwrap error when scheduling symbol graph task

41302ce

ahoppen reviewed Jul 7, 2026

View reviewed changes

Padmashree06 added 4 commits July 15, 2026 16:53

Merge branch 'swiftlang:main' into emiting-symbol-graph

1da0746

Address review feedback & add go-to-definition for docc Block comments

bb06bfd

Switch to next language service on return nil

dc92f9c

Resolve clicked path component in DocC symbol links

f318e62

Padmashree06 force-pushed the emiting-symbol-graph branch from 969ba7d to f318e62 Compare July 17, 2026 07:00

ahoppen reviewed Jul 17, 2026

View reviewed changes

Add symbolGraphUpToDateTracker & remove unwanted transformations

c1fa8c8

Padmashree06 force-pushed the emiting-symbol-graph branch from f88dc14 to c1fa8c8 Compare July 19, 2026 09:22

ahoppen mentioned this pull request Jul 23, 2026

Add diagnostics for missing/broken Symbol Links in DocC comments, Tutorial and Markdown files #2729

Open

Resolve DocC go-to-definition via index instead of full symbol graph …

e1eeb0b

…generation

		}

		package func definition(_ req: DefinitionRequest) async throws -> LocationsOrLocationLinksResponse? {

		package func symbolInfo(_ req: SymbolInfoRequest) async throws -> [SymbolDetails] {
		return []


		private let updateSymbolGraphIDForLogging = Atomic<UInt32>(1)

		package struct UpdateSymbolGraphTaskDescription: IndexTaskDescription {

Trial	Time taken for SymbolGraph generation (ms)	Time taken for Indexing (ms)	Overhead percentage (%)	Performance cost
1	159,494	63,872.3	249.70	2.5×
2	171,342	82,867.8	206.76	2×
3	352,348	146,415	240.65	2.4×

		if !baseArgs.contains("-wmo") && !baseArgs.contains("-whole-module-optimization") {
		baseArgs.append("-wmo")

Uh oh!

Conversation

Padmashree06 commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

User Impact

Uh oh!

matthewbastien commented Jun 26, 2026

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Padmashree06 Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewbastien commented Jul 9, 2026

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Padmashree06 commented Jun 20, 2026 •

edited

Loading

Padmashree06 Jul 20, 2026 •

edited

Loading

Padmashree06 Jul 19, 2026 •

edited

Loading

Padmashree06 Jul 18, 2026 •

edited

Loading