Building a Free Code Assistant

Prince Addo, Verrazzano Class of 2024, completed major in Computer Science and minor in Mathematics

For my capstone I worked on a free and open-source AI(Artificial Intelligence) code assistant. More specifically, I built a free and open-source neovim plugin called ollama-pilot.nvim that uses the ollama framework for open-source model integration. Although AI code assistant’s tools have quickly become popular amongst programmers, their close-source nature has been contentious. My project is novel in that it is the first implementation of a completely open-source code assistant.

I’ve had a real interest in hacking ever since taking computer science seriously after graduating high school. The idea of creating clever tools to break into systems piqued my interest; it was a mixture of technology and art, which I was exploring at the time. In order to successfully break into a system, like a true hacker, you must first understand the ins-and-outs of a system; only then can you creatively break it—like Picasso once said, “learn the rules like a pro, so you can break them like an artist.” Although I was never able to fully pursue hacking, this interest led to me learning about software development, Linux, and most importantly, open-source.

To me, open-source is a positive and peaceful place; there exists bad actors and people with ulterior motives, but similar to capitalism or blockchain algorithms, unless the majority of the system consists of bad actors, the system will function as intended. In computer systems, the source-code is the truth, therefore if someone hides or obfuscates the source-code then they might be lying—and one can argue that they probably are. As the name implies, open-source is about making the source-code open to the public to view, therefore it is the ultimate expression of honesty that can be made. If the source-code is available, and you want to know whether a program is not doing anything malicious, you can just read the code. If it’s not available, you have to operate on the trust-me-bro principle, which is infamous for being precarious in the computer industry.

Open-source is also a place to share and collaborate on projects with like-minded people. This builds confidence and a sense of community, which is important since the practice of programming is often isolating. I’ve met some of my closest friends working on open-source projects. Open-source can be taken further by making it free; the term free is not free as in price, all open-source projects are by nature free, but as in the freedom for anybody to modify the source-code in whichever way they want. If any security concern was introduced, instead of being only able to detect it, they would be able to remove it.

I came across Vim, a free and open-source project, really early in my programming journey. Once you’re in the realm of Linux, Vim is bound to come across your radar. It took me a while to appreciate it, but as I got more proficient, I gained more respect for it. It always felt a little too difficult, so I was never able to fully dive into it until NeoVim came out. NeoVim takes Vim and adds more features to, features which for one, make it more accessible to new programmers. On top of that, package managers like lazy.nvim that recently came out have made the experience of plugin management a lot easier and intuitive.

After the release of ChatGPT and the subsequent release of all things AI related, I started exploring AI and AI tools and began using my first code assistant called Codeium. I loved using it because it simplified the development process, but I couldn’t really trust it. I knew that my data was being sent to the Codeium servers and the code for those servers are not open-source, therefore who knows what they are doing with it. I had to shut it off and I eventually just stopped using it. I looked for some solution, but there was none, so I endeavored to build one, which is where this project comes in. Open-source AI is at its infancy and has several problems. One is that it requires an expensive computer to run it in a reasonable amount of time—or sometimes run at all. Another problem is that it is difficult to host. The latter problem was solved once I found the ollama framework, which is essentially a framework that allows you to host open-source models. The former problem is much harder, but will probably be fixed in the coming years. Once the major roadblocks were gone all that was left to do was architect and write the code. This proved to be very difficult as the learning curve was steep, but I committed myself to breaking the project into components and tackling each component in the development process. As of this writing, I have not completed the project, but I plan on getting at least an alpha or beta version out there.