In this era of Computers, almost everyone is aware of Large Language Models (LLM) and all the magic that can be done with them. This field of AI is growing rapidly with new and better models being released by top technology companies in the world.
And here comes Google Inc. which recently released Gemma. If you want to learn and be updated on the model's key features, benefits, project nature, capabilities, comparison with other models, and what the future holds for it, this is the right article for you.
Let’s get started:
What is Gemma?
Gemma is a family of Open Models based on Google Deepmind’s Gemini Research and Technology. It is a family of lightweight, state-of-the-art open models built on the research and technology used to create the Gemini models.
It was released in 2 sizes by Google Deepmind’s Gemma Team in 2023, that is, the 2B parameter and 7B parameter Gemma models.
Key features and benefits
These models are claimed to have been trained on 6 Trillion tokens of text using the similar architecture and data used to train the prominent Gemini AI. The 7B parameter model is efficient for deployment and development on GPU and TPU while the 2B parameter model is for CPU and on-device applications.
Let’s dive into the benefits of these models:
They are lightweight compared to other language models like LaMDA and Megatron. This means they do require less computational power making them accessible on personal computers and cloud platforms with fewer resources. Hence smaller teams and individuals can now access this technology.
Since the code for these models is open for use by the public, anyone can access or experiment with it or even modify it. This encourages collaboration among various AI teams willing to develop and impact the world with amazing technologies.
Despite being small, Gemma models achieve competitive performance on many things like generating text, translating, and answering questions thus they can be used in real-world applications.
These models can be fine-tuned for specific tasks making them more versatile and powerful for tailored applications.
Open-source nature and Accessibility
These models are open-source and can be accessed by any developer willing to use them or contribute through bug reports, pull requests, and other kinds of contributions.
Follow this link for the Github repository where you can access these models for use in your projects: Gemma Repository
Ethical Considerations
In 2021, Weidinger said, “In addition to bringing benefits to the AI development ecosystem, we are aware that malicious uses of LLMs such as the creation of deep fakes, AI-generated disinformation, and illegal and disturbing material can cause harm on both an individual and institutional levels.” So you might be asking yourself, what have the Gemma models brought in because these LLMs can sometimes negatively impact the community?
According to the release report of the Gemma Team, they are trying to solve the malicious use of the model by safeguarding against these risks through filtering and measuring bias in pre-training data in line with the Gemini approach thus assessing safety through standardized AI safety benchmarks, internal red teaming to understand better the risks involved with the external use of Gemma, and subjecting the models to rigorous ethics and safety evaluation.
Technical Specifications
The Gemma models can run on CPU, GPU, and TPU. For the GPU, an 8GB+ RAM size is recommended for the 2B model and 24GB+ RAM for the 7B model.
Capabilities and Applications
These models improve performance on a broader range and can be applied in dialogue-based systems, reasoning applications, mathematics, and code generation
Comparing Gemma to other Language models
The most prominent competitors to Gemma models are the LLaMA models, the Mistral model, and the LaMDA and Megatron models.
According to the technical report released by the Gemma developer, they claim to have tested the various models based on Question Answering, Reasoning, Math/Science, and Coding.
Below is an image showing the performance of each model based on the above features:
Some of the test results of the model depending on its memory of data sources and degree of approximation and exactness are also shown in the images below:
Conclusion
The future of AI seems to be drifting further towards the unforeseen, from Large Language Models like ChatGPT to lightweight models like the Gemma models which can now run on personal computers and the cloud.
If you are a developer or a researcher in the field of AI and Technology, it is high time you dived into an exploration of what the next future is always bringing for us.
For additional reading about the Gemma models and how you can run them on your computer and use them for various tasks, check the resources provided below:
More of the resources are below:
Safe journey to the future --->
Download the Gemma Models from Kaggle here: Kaggle Gemma Models