I would like to know llm.c!
Reliability of This Article
by Our Founder/CEO&CTO Hiroyuki Chishiro
- He has been involved in 12 years of research on real-time systems.
- He teaches OS (Linux kernel) in English at the University of Tokyo.
- From September 2012 to August 2013, he was a visiting researcher at the Department of Computer Science, the University of North Carolina at Chapel Hill (UNC), Chapel Hill, North Carolina, United States. He has been involved in research and development of real-time Linux in C language.
- He has experienced in more than 15 years of programming languages: C/C++, Python, Solidity/Vyper, Java, Ruby, Go, Rust, D, HTML/CSS/JS/PHP, MATLAB, Verse (UEFN), Assembler (x64, ARM).
- While a faculty member at the University of Tokyo, he developed the "Extension of LLVM Compiler" in C++ language and his own real-time OS "Mcube Kernel" in C language, which he published as open source on GitHub.
- In January 2020-Present, he is CTO of Guarantee Happiness LLC, Chapel Hill, North Carolina, United States, in charge of e-commerce site development and web/social network marketing. In June 2022-Present, he is CEO&CTO of Japanese Tar Heel, Inc. in Chapel Hill, North Carolina, United States.
- We have been engaged in disseminating useful information on AI and Crypto (Web3).
- We have written more than 20 articles on AI including AI chatbots such as ChatGPT, Auto-GPT, Gemini (formerly Bard). He has experience in contract work as a prompt engineer, manager, and quality assurance (QA) for several companies in San Francisco, United States (Silicon Valley in the broadest sense of the word).
- We have written more than 40 articles on cryptocurrency (including smart contract programming). He has experience as an outsourced translator of English articles on cryptocurrency into Japanese for a company in London, England.
You can learn from us.
If you would like to know the recommended job sites for AI Engineers, please click the following.
If you would like to know the recommended job sites for Prompt Engineers, please click the following.
Table of Contents
What is llm.c?
llm.c is a framework for training Large Language Models (LLMs) using pure C and CUDA without PyTorch or cPython.
By using llm.c, you can train LLMs about 7% faster than with PyTorch.
Since "llm" in "llm.c" is a Large Language Model and ".c" is an extension for C source files, the name of the framework like the filename of C source files is good!
As of July 2024, llm.c will focus on pretraining (especially GPT-2 and GPT-3 reproduction) and parallel PyTorch reference implementation in train_gpt2.py.
In the future, llm.c may be used in state-of-the-art ChatGPT training.
If you want to know how to start ChatGPT and how to use it, please click the following.
llm.c is available as open source on GitHub.
|
1 |
$ git clone https://github.com/karpathy/llm.c |
The open source license is MIT License.
Projects to Run llm.c in Other Languages and Environments
Projects to run llm.c in other languages and environments is underway.
More details can be found on the official llm.c page.
The projects as of July 2024 are as follows.
- AMD support
- C#
- CUDA C++
- Go
- Java
- Metal
- Mojo
- OpenCL
- Rust
- Swift
- Zig
Introductory Videos of llm.c
These are introductory videos of llm.c.
Summary
We introduced "llm.c," a framework for training Large Language Models (LLMs) in C language.
We hope that llm.c will lead to the widespread use of C language in generative AI!
If you would like to know the recommended job sites for AI Engineers, please click the following.
If you would like to know the recommended job sites for Prompt Engineers, please click the following.