Editor Integration in Carbon Language
I'm Manmeet Singh, and I took part in Google Summer of Code 2023, contributing to the Editor Integration for the Carbon Language.
In the realm of software development, a seamless editor experience is of paramount importance. This Google Summer of Code project was to improve the developer experience for Carbon programming language enthusiasts by integrating Carbon into various editors and IDEs. This post takes you on a journey through the Editor Integration project for Carbon, highlighting its accomplishments, current state, challenges, and future prospects.
Project Goals and Implementation
The project's overarching goal was to facilitate smoother coding for Carbon developers by integrating Carbon language support into popular editors and IDEs. To achieve this, I divided the project into three distinct components: Tree-sitter grammar implementation, refining TextMate syntax highlighting, and developing a Language Server.
Tree-sitter Grammar
One of the core elements of this project was providing a Tree-sitter grammar for Carbon. Tree-sitter is a parser generator that provides robust parsing capabilities. I started implementing the Tree-sitter grammar. Mentor's reviews were very helpful to find edge cases and incorrect grammar. One hard case was distinguishing between a postfix * and infix *. This is already handled by Symbolic Tokens. Tree-sitter grammar is tested against existing explorer test data to ensure its completeness.
This achievement marked a significant stride, as it allows Carbon code to be fully parsed by editors. Notably, the Tree-sitter is used in the Neovim editor for syntax highlighting. Tree-sitter grammar can also be used for structural code navigation. Additionally, Treesitter is also supported by Emacs, although an Emacs plugin for Carbon is not implemented yet.
TextMate Syntax Highlighting
TextMate is a regex-based syntax highlighting approach. Carbon already had basic support for TextMate syntax highlighting. As part of this project, I improved the syntax highlighting. I updated keywords to align with the evolving Carbon language. Moreover, I added a Visual Studio Code extension that utilizes the TextMate syntax file for improved syntax highlighting. Currently, TextMate syntax highlighting is functional in both Visual Studio Code and JetBrains IDEs.
Language Server Protocol Implementation
The third part of the project involved the implementation of a Language Server for Carbon. A Language Server is meant to provide the language-specific smarts and communicate with development tools over a protocol that enables inter-process communication. The idea behind the Language Server Protocol (LSP) is to standardize the protocol for how such servers and development tools communicate. This way, a single Language Server can be reused in multiple development tools.
I developed a basic language server for Carbon. Language server uses the Carbon toolchain. I reused language server protocol code and its helpers from clangd to avoid reimplementation. Language server supporting code outlining. I implemented Carbon language server clients for VSCode and Neovim.
Challenges and Learning
This project was not without its challenges. The integration of Tree-sitter grammar required grappling with Bazel integration. Additionally, for the Language Server, C++ was a learning curve. However, these challenges served as valuable learning opportunities, imparting expertise that will undoubtedly prove invaluable in future projects.
The Current State and What Lies Ahead
As it stands, the Editor Integration project for Carbon has made impressive strides. Developers using Neovim, VSCode, and JetBrains IDEs have an enhanced experience. The Tree-sitter grammar, TextMate syntax highlighting, and Language Server have enriched these editors with Carbon support. Detailed instructions can be found in utils/README.md for setting up Editor integration in your favorite editor.
Looking ahead, there are exciting prospects to build on these accomplishments. Future work includes expanding the language server capabilities to encompass features like goto definition, find references, variable renaming, and auto-complete.
Conclusion
In closing, a heartfelt thank you to the mentors for their meticulous reviews and guidance throughout the project. The community's support has been instrumental in shaping the success of this endeavor.