PinnedOne year later… ChatGPT vs Open source LLMsSince ChatGPT launched on November 30, 2022. the race to develop an open source model that can match ChatGPT like experience started… Almost a year later, can open source community developed models catching up ChatGPT in term of user experience? People like ChatGPT because that enables users to refine and…ChatGPT8 min readChatGPT8 min read
PinnedTest Drive Fast Inference via Speculative Decoding without QuantizationLarge autoregressive models, notably large Transformers (Vaswani et al., 2017), are much more capable than smaller models, as is evidenced countless times in recent year. …Fast Inference13 min readFast Inference13 min read
PinnedFinetune LLM to convert a receipt image to json or xmlMotivation convert a receipt image to a JSON or XML object is to make it easier to store, analyze, and manipulate the data in a structured and organized way. Both JSON (JavaScript Object Notation) and XML (Extensible Markup Language) are lightweight data interchange formats that are easy for humans to…Llm Finetune10 min readLlm Finetune10 min read
PinnedFine-tuning LLM Model (Llama2) for inverse information generation(reverse-thinking) The hype for Large Language Models (LLMs) came from ChatGPT. These models are so good at understanding and generating text that it shocked people, myself included. Training a Large Language Model is a very costly and long process with a lot of hardware issues. The release of Llama 2…Llm7 min readLlm7 min read
Oct 28Practical Techniques to constraint LLM output in JSON formatJSON is one of the most widely used data interchange formats globally, supporting all our requirements. When building an AI-powered app, engineers inevitably need to integrate the outputs of a Large Language Model (LLM) into their codebase. By instructing the LLM to use a specific syntax or schema and then…Llm Json Output12 min readLlm Json Output12 min read
Oct 26Why Offline AI-Voice Assistant is the FutureOur voice is the most beautiful and natural communication method we use every day. This is part-one of the blog on why offline AI-voice assistants are the future. …Voice Assistant6 min readVoice Assistant6 min read
Oct 11Improve model output quality with Reinforcement learning from human feedback (RLHF) —Classic exampleOur goal is to productionize a reliable LLM model that consistently produces results that adhere to the established business practices, rules, intentions, and security standards. However, implementing an LLM solution presents challenges, particularly when it comes to controlling the model to generate predictable output that required specific knowledge domain. While…Rlhf6 min readRlhf6 min read
Oct 9Implement Knowledge Distillation to compress large models into lightweight versionDeveloping large models improve state-of-the-art performance, but deploying such big models is not straightforward nor very feasible. specially for edge devices. due resource limitation or cost makes unpractical for production deployment. As the result often leads to the development of very big deep learning models that yield very good accuracy…Knowledge Distillation5 min readKnowledge Distillation5 min read
Oct 7TinyLlama Colorist - fine-tuned with Color datasetRecently a project caught my attention is the TinyLlama project which aims to pretrain a 1.1B Llama model on 3 trillion tokens. It adopted exactly the same architecture and tokenizer as Llama 2. With only 1.1B parameters, this compactness allows it to cater to a multitude of applications demanding a…Tinyllama4 min readTinyllama4 min read
Aug 13Build a specialized Llma-2 model for product brand recommendationWhile LLaMA-2 is a better model in terms of quality than LLaMA, has double the context length, and is commercially viable, it typically won’t work well out of the box for your specific ML task since it was trained on general text data from the web during the pre-training stage. …Llm4 min readLlm4 min read
