New LSP language service supporting Swift and C-family languages for any editor and platform (original) (raw)

October 15, 2018, 9:00pm 1

Hey all,

At Apple we are making it a priority to support high-quality tooling for all Swift developers, including those working on non-Apple platforms. We want to collaborate with the open-source community and focus our efforts on building common infrastructure that can be shared by Xcode and other editors and platforms.

To that end, I'm excited to announce that we are going to start a new open-source project for a Swift and C-family language service based on the Language Server Protocol. We've chosen to adopt LSP so we can benefit from its active community and wide adoption across other editors and platforms. This means that Visual Studio Code, Atom, Sublime Text, or whatever your favorite editor happens to be, can use the same service as Xcode, and any improvements we make to the service will benefit them all.

This new LSP-based language service will support:

We are also going to make the necessary changes for Xcode to use this new LSP-based language service. Similar to how you can load an OSS toolchain in Xcode to try it out now, in the future you will be able to load a build of the OSS LSP service in your installed Xcode and test any fixes or enhancements you make.

We will try to stick to the common set of LSP requests, but for functionality in Xcode that is not currently supported by LSP, we will add extensions to the protocol. If these extensions can be generally useful for other editors as well, we will propose adding them to the LSP specification.

Given that we want to offer first-class support for other editors and platforms, we will aim for the new LSP service to have great support for SwiftPM projects. We will also be open to contributions for supporting other systems as well, like Bazel.

We should have a project repository up within a month, and in the meantime I'd be happy to hear any feedback or questions you may have!

duan (Daniel Duan) October 15, 2018, 9:39pm 2

As a daily Swift + Vim user I'm super excited to see things moving in this direction. To that end I have a couple of questions on the technical side:

  1. What would the relationship between this new project (we need an official name for it :wink:) and SourceKit, when it comes to syntax highlighting? Is SwiftSyntax part of it?
  2. For indexing/code completion/jump-to-definition: will this project replace Xcode's current implementation, or exist in parallel?

Daniel_Martin (Daniel Martín) October 15, 2018, 9:44pm 3

Oh, that's great! I was starting to develop a Swift LSP server myself, but I'd probably contribute to this project instead.

One question I have is about the C/C++/ObjC part of this project. How will this project fit with clangd, which is already part of the Clang repository? Are you going to reuse code, start from scratch...?

Jens October 15, 2018, 9:45pm 4

Could this (among other more important things) also (finally) bring an improvement to Xcode's automatic indentation with it?

An example

Here's what it currently thinks this code should be indented like (even if you try to Editor -> Structure -> Re-Indent):

func f(_ v: Int) -> Int {
    guard
        v > 0
        else {
            return 0
    }
    let x = "abc" +
    "def"
    let y =
    123
    return x.count + y +
    v
}

I think most people would expect:

func f(_ v: Int) -> Int {
    guard
        v > 0
    else {
        return 0
    }
    let x = "abc" +
        "def"
    let y =
        123
    return x.count + y +
        v
}

blangmuir (Ben Langmuir) October 15, 2018, 9:50pm 5

The LSP service will use sourcekitd to get its swift-specific support - ie. highlighting, code-completion, etc. I'd love to use libSyntax to implement syntactic highlighting, but that would be an implementation detail. I expect we will add an (extension) API to provide the syntax tree though.

The intention is for this project to supercede the indexer we have today when it's ready.

blangmuir (Ben Langmuir) October 15, 2018, 9:54pm 6

Our service will use clangd :smile: The idea is that we will load clangd + sourcekitd from a toolchain to provide the language-specific functionality for e.g. code-completion.

blangmuir (Ben Langmuir) October 15, 2018, 9:56pm 7

This should be fixable regardless of whether it's in the new service or not. Have you filed a bug for this one?

akyrtzi (Argyrios Kyrtzidis) October 15, 2018, 10:35pm 8

To add to what Ben mentioned, the new service will be written in Swift so it could use SwiftSyntax and any other functionality written on top of SwiftSyntax, like formatting.

gmittertreiner (Gwen Mittertreiner) October 16, 2018, 12:00am 9

This is excellent! I had started implementing exactly this out of necessity at work for my team (DevX related tools at Facebook), but it seems like a good idea to join efforts.

How far along is the project right now? Would you mind us being included in some design talks? This aligns closely with a number of goals we have. We'd definitely be interested in spending time contributing to this as soon as possible. Would someone by chance be at the LLVM meeting this week?

akyrtzi (Argyrios Kyrtzidis) October 16, 2018, 12:42am 10

I will be there, along with @blangmuir and @Nathan_Hawes. Jason, I've sent you a direct message. If anyone else will be at LLVM Dev meeting, and wants to chat about the LSP service, send me a message.

allevato (Tony Allevato) October 16, 2018, 12:51am 11

Google is also very interested in contributing, as we've done a fair amount of work in this area to meet some of our internal needs (Bazel support is a big one for us, but general functionality as well). We're excited to see this moving forward!

DevAndArtist (Adrian Meister) October 16, 2018, 4:48am 12

I‘m thrilled by this announcement. You mentioned that this project itself will use Swift as its main language. Which coding style will the project follow? I‘d like to note that I really like the consistent way Swift code is written in stdlib compared to the inconsistent corelibs repo.

beefon (Vladislav Alekseev) October 16, 2018, 5:31am 13

This is great. I wonder how LSP could help build systems like Bazel and Buck? Could you provide any example of the feature that can be added into the build systems? Will LSP somehow improve llbuild?

samdeane (Sam Deane) October 16, 2018, 10:31am 14

This is great news.

I've been distracted from the atom integration that I was attempting, but this development should help to nudge me back into working on it.

ankit_goel (ankit goel) October 16, 2018, 10:42am 15

I am very interested in contributing to this. Can't wait for the repo to be up!

RUSshy (Ru Sshy) October 16, 2018, 11:51am 16

Thanks a lot !! This is big news !

matklad (Aleksey Kladov) October 16, 2018, 1:06pm 17

:+1:, libSyntax would be an awesome foundation for close-to-concrete-syntax IDE features

we will add extensions to the protocol.

You might want to take a look at the Dart's protocol for inspiration, it seems to handle some stuff better than LSP.

allevato (Tony Allevato) October 16, 2018, 1:23pm 18

The general idea is that a language server needs to know the compilation commands that are used to build the modules used by a project (SourceKit does as well), for things like inter-module dependencies or taking the right branch of #if blocks based on the -D flags or target machine, and so forth. Since SwiftPM describes its build in one way and Bazel describes it in a different way, the LS needs to understand how to get that information from those systems.

So for example, if you had a VS Code workspace open and the LS detected that you had a typical SwiftPM directory structure, it could ask SwiftPM to provide those flags. If it detected that the project was a Bazel workspace structure, it could ask Bazel instead, and so forth.

slinberg (Steve Linberg) October 16, 2018, 1:53pm 19

At Apple we are making it a priority to support high-quality tooling for all Swift developers, including those working on non-Apple platforms

Super. Any progress on non-Ubuntu Linux, like RHEL or CentOS? "yum install swift-dev"?

Schniz (Gal Schlezinger) October 16, 2018, 2:26pm 20

Hey @duan, maybe unrelated to the thread, but can you share with me your vim & swift configuration?
I wonder if I can be happy in Vim in Swift like I am with any other language :smile_cat:

Are you doing it for backend/cli dev or for iOS/macOS/tvOS development?