Recently, Amazon Cloud Technology announced a machine learning-powered service that helps improve developer productivity by generating code suggestions based on their comments in natural language and their code in an integrated development environment. Work efficiency. The service, called Amazon CodeWhisprer, is still in preview and free to use. The service is similar to the GitHub copilot Microsoft launched last year.
Amazon Cloud Technology Developer Community provides developers with global development technology resources. There are technical documents, development cases, technical columns, training videos, activities and competitions, etc. Help Chinese developers connect with the world's most cutting-edge technologies, ideas, and projects, and recommend outstanding Chinese developers or technologies to the global cloud community. If you haven’t followed/collected it yet, please don’t swipe it hastily when you see this, click Here Let it become your technical treasure trove!
Over the past few months, I've had the opportunity to experiment with this new service in several use cases. As a machine learning (ML) developer, I have the advantage of leveraging ML to help develop ML solutions. Therefore, I wrote some observations after early access to this service. Additionally, I'm providing specific suggestions on how to make it smarter and more accessible.
Service in action
The service provides real-time code suggestions based on comments in the code editor and previous code in the same document. The service can suggest line completion or complete blocks of code (for example, methods).
On Visual Studio, there are some handy shortcuts that make using services easier. When the extension is enabled, the service provides online reasoning similar to the autocomplete functionality supported by many IDEs. However, users can click (Alt+C) to view recommendations without waiting for a response.
Here is an example of writing the famous binary search method
Interestingly, the service may suggest multiple snippets that can be easily navigated (using left/right arrows) to select the most appropriate recommendation.
Amazon CodeWhisprer is like a companion trying to whisper the right code into your ear. Therefore, it's a very fancy and super descriptive name. Great job on naming services.
Digging deeper, how do you make the most of the service?
AI Code Companion is a powerful tool that makes developers more productive. Although some believe that such tools may replace developers in the future, it is too early to tell because the service is like any other: Garbage in Garbage out< a i=2>. That is, it depends heavily on the input required to return good results. Here's an example of how input quality completely affects output quality.
Here, the description provided is vague and there are no clear requirements, so after waiting for a relatively long time, the output is a confusing import.
As the input description becomes clearer, the output gets better, as shown below, which is a similar but clearer problem.
Furthermore, as users add more context and developers write more code, the quality of recommendations improves significantly. For example, expect faster and more personalized results when working on a project than when working on an isolated task on the same document or early in the project when context is still insufficient.
Despite this, the service is not expected to return useful answers for the notorious custom tasks. Below is an example of the same binary search problem, but with a slightly modified input format.
Apparently the engine is unable to understand a slight modification to the problem (i.e. allowing duplicate elements) and still produce the same code as suggested previously.
Could the service be better?
Since the service is still in preview, expect to encounter a number of shortcomings. Below is a curated list of actions that can make your service better.
Reasoning speed:
As might be noted in the example above, the service takes a lot of time to come up with recommendations. I believe there is a lot of room for improvement in this area.
Consistency and real-time:
The service promises to provide real-time suggestions as developers write code. However, real-time suggestions may not give any output at a specific moment. Surprisingly, pressing the (Alt+C) shortcut returns a working solution without changing anything (i.e. instantly at the same time).
End user customization:
The recommendation engine under the hood uses a huge code base derived from many source codes written for different purposes. It is reasonable to enable more customization for the sources accepted by certain projects.
Additionally, it may be beneficial to predict code based on the project topic. For example, machine learning development is completely different from developing mobile apps.
As another example, a user may want to work on a project that requires multiple code blocks to be designed and aggregated. In other projects, line completion may need to be prioritized over blocking proposals.
The list of custom examples is very large and requires careful design.
Solution ranking:
Suggesting multiple solutions is a great feature. However, in practice, the ranking of these solutions is not optimal and users need to browse through all solutions to find the right suggestion. This can be tedious and reduces overall productivity.
Question customization:
The engine effectively understands common problems found in the training corpus. However, it is more difficult to adapt to new challenges of the same problem.
in conclusion
All in all, Amazon CodeWhisprer (and AI code companions in general) is not a magic bullet that can solve all problems after all. However, it is a great tool for making developers more productive by focusing on the right problems instead of tedious repetitive tasks.
To get the most out of Amazon CodeWhisprer (and AI code companions in general), the following may help achieve your desired goals:
- Concise comment: The clearer the input task, the higher the probability of achieving quality results.
- Unified Project: An artificial intelligence engine collects information from the entire document. Therefore, it continuously enriches the context. Therefore, it would be more beneficial to use it for tasks that have connections in some way.
- Avoid advanced custom questions: The more unpopular a question is, the higher the chance that it will not return any useful answers.
