Apple releases the large language model “OpenELM” to insure the transparency in research on Large language models at advance level. Instead of using prior LLM models, OpenELM introduce complete framework for training large language models by using public dataset and it has included pre-training configurations, checkpoints while in on the other hand others has only private pre train dataset.
Apple OpenELM LLM also focus to release and provide MLX library to train and fine tune datasets on Apple devices, This is huge and more reliable for research and developer communities of LLM and AI.
Which is Better OpenELM vs OLMo
OLMo has trained on 1.2 billion parameters but OpenELM trained with only 1.1 billion parameters but it has more power than OLMo, and require less training data.
OpenELM Uses Public Data For Pre-Traning
OpenELm is trained on public dataset like refined web, PILE(deduplicated), RedPajama subset, Dolma subset, token used in OpenELM is 1.8 trillion.
Total number of OpenELM models are 8 including ELM 1.1 billion…
- OpenELM 270M
- OpenELM 450M
- OpenELM 1.1 Billion
- OpenELM 3M
- OpenELM 270M-Instruct
- OpenELM 450M-Instruct
- OpenELM 1.1 B-Instruct
- OpenELM 3B-Instruct
OpenELM’s are the family of opence source language models, to ensure accuracy of model Apple pre-train OpenELM’s using CoreNet Library(CoreNet is a library for training neural networks).
OpenaELM is publicly available on LLM hosting platform “Apple Hugging Face Page” here is the link of HuggneFace OpenaELM Model.
To read full research paper must visit here / pdf.
While make it opensource it helps lot much to AI community and also Apple, apple not openly announces that they are linking it to their Operating system but rumours say yes.
after hand down to open source of elm model apple will be make sure to integrate this with their devices. because of its reliability and lightweight llms and also serverless. HELM do not need GPU servers to run it.
