How useful have the GitLab Duo AI tools been so far?

They are quite similar to OpenAI's tools. The code generation on a larger scale, like solving an entire problem, isn't as effective as some might hope. It generates a lot of code, often bloating it unnecessarily. For example, a young developer here was solving a problem he had never encountered before. He quickly ramped up and generated a few hundred lines of code that actually did something. I sat with him, asked a few questions he couldn't answer, and we manually trimmed it down, removing unnecessary parts like error handling that couldn't occur. We ended up with 10 lines of code, so it had inflated the code by about 50 times.

How useful have the GitLab Duo AI tools been so far?

This happens because it learns from other trained codebases where people handle errors or perform tasks in certain ways. While there is a lot of useful content on GitLab and GitHub, which they use to train their models, there is also a lot of irrelevant information. Currently, there is no filtering; they simply feed it everything. The question is whether there is more irrelevant content or more useful information.

Do you use GitLab, then this other tool for niche cases?

GitLab's decision to allow other tools to upload scanning results in JSON format was a genius move. This change, which happened one or two years ago, or maybe even three, also allowed third-party hooks to block merging certain functionalities. For instance, you could set up SonarQube or other third-party tools to block a merge if they detected a vulnerability that GitLab missed.