Various RWKV related links

The following are links to various RWKV community projects (usually hosted on GitHub) for specific use cases or references.

Warning

Marked as (May be Outdated) indicates that the project has been archived or has not been updated for a long time. However, even if some projects are outdated, their implementation ideas can still be used as a reference for development.

In addition, we would like to thank the developers who have contributed to the RWKV ecosystem, whether they are still active in the RWKV community or not.

RWKV Fine-tuning/Training Tools

Fine-tuning Projects

  • (Recommended) RWKV-PEFT: An efficient fine-tuning repository for RWKV, which includes various fine-tuning methods such as LoRA, Pissa, Lisa, State, etc.
  • RWKV-PEFT-Simple: A simplified version of RWKV-PEFT, providing convenient scripts and fine-tuning instructions
  • (Under Development) RWKV-LM-RLHF: An RWKV reinforcement learning (RLHF) toolkit, including SFT, alignment (DPO, ORPO), etc.
  • RWKV-LM-RLHF-DPO: The DPO implementation (Direct Preference Optimization) of RWKV
  • (May be Outdated) RWKV-LoRA: The LoRA fine-tuning implementation of RWKV, which does not support fine-tuning RWKV-6 or later versions.

Training Projects

  • RWKV-infctx-trainer: An RWKV infinite context trainer that supports training contexts of 10k length and above
  • RWKV-Ouroboros: An rwkv-trainer project based on the API, which supports alternating training and inference
  • nanoRWKV: The nanoGPT-style implementation of the RWKV model
  • RWKV_LM_EXT: Realizes the functional extension of RWKV, including sequence classification/embedding/peft/cross-encoder/dual-encoder/multimodal, etc.
  • chunkRWKV6: Optimizes the prefill and training speed of RWKV using block parallelism

RWKV Inference Frameworks

rwkv.cpp

  • rwkv.cpp: Inferences the RWKV model on the CPU, supporting FP16, quantized INT4, INT5, and INT8 inferences

Pytorch Native Implementation

  • RWKV_Explained: The Pytorch implementation and code explanation of RWKV
  • RWKV_Pytorch: The inference framework of the RWKV model implemented purely with native Pytorch
  • rwkv-kit: The inference framework of the RWKV model, supporting various functions such as batch inference and parallel inference training

WebGPU Implementation

  • web-rwkv: Implements the inference of the RWKV model in pure WebGPU/Rust

candle Implementation

  • candle-rwkv: Implements the inference of the RWKV model (supporting quantization) in the minimalist machine learning framework candle of Rust

Keras Implementation

  • RWKV6-Keras: The Keras implementation of the RWKV model, supporting both training and inference

JAX Implementation

  • rwkv-jax: The JAX implementation of the RWKV language model, suitable for TPU training
  • (May be Outdated) RWKV-LM-jax
  • (May be Outdated) tpu_rwkv

Mobile Inference

  • (Under Development) rwkv-mobile: The RWKV mobile inference engine
  • rwkv-qualcomm: Infers the RWKV model using the Qualcomm AI Engine Direct SDK

Other Inference Engines Supporting RWKV

  • RWKV-Infer: Performs RWKV-V6 inference using FLA, supporting inference by combining multiple states
  • MLC LLM: A machine learning compiler and high-performance deployment engine, supporting the RWKV model

RAG (Retrieval-Augmented Generation) System

  • RWKV-RAG: A one-click RAG deployment system based on the RWKV model, which can easily build and manage a local knowledge base, and also provides a question-answering robot based on the local knowledge base and the one-click fine-tuning function of RWKV

Chatbots/Inference API Servers

  • (Chinese, Recommended) RWKV runner: RWKV Runner is a management and startup tool for the RWKV model, with a user-friendly GUI interface, supporting both training and inference
  • (Chinese, Recommended) AI00 RWKV server: The Ai00 Server is an inference API server for the RWKV model based on the web-rwkv inference engine
  • (Chinese) Wenda web UI: An LLM calling platform for content generation in specific environments, supporting the RWKV model
  • (Chinese) role play chatbot: A role-playing webui based on RWKV, made using Gradio
  • Easy_RWKV_webui: An online chat room for the RWKV model based on PyWebIO
  • Mini Model Daemon: An inference tool for the RWKV model based on the web-rwkv inference backend
  • LocalAI: An open-source alternative to OpenAI, supporting RWKV
  • GPT Academic: Provides practical interactive interfaces for large language models such as GPT/GLM, supporting RWKV
  • LLMFarm: Uses the GGML library to use large language models offline on iOS and MacOS, supporting RWKV

RWKV Benchmark Tests

RWKV Multimodal

Tips

Multimodal refers to tasks other than text tasks.

Any Modality

  • (Under Development) WorldRWKV: Uses the pure RWKV7 architecture to implement the training and inference of any modality
  • RwkvCompress: An image compression model based on RWKV, which can efficiently compress and reconstruct images
  • VisualRWKV: A visual language model based on RWKV, which can handle visual tasks
  • Vision-RWKV: A visual perception model based on RWKV, which can smoothly process high-resolution images
  • Diffusion-RWKV: A model for image generation tasks based on RWKV, which is good at handling high-resolution images
  • RWKV-CLIP: The CLIP (visual-language representation learning) model driven by RWKV
  • RWKV-SAM: The image segmentation method "RWKV-SAM" based on RWKV
  • PointRWKV: A 3D point cloud learning framework based on RWKV
  • Restore-RWKV: Uses RWKV for efficient and effective medical image restoration (PyTorch implementation)
  • LION: A Linear Group RNN for 3D object detection in point clouds (supporting RWKV)
  • LineRWKV: LineRWKV is a method for lossless and lossy compression of hyperspectral images
  • RWKV-ASR: Uses the pre-trained RWKV language model for speech recognition

Time Series

  • BlackGoose Rimer: A pure RWKV time series task model that surpasses the Transformer in both speed and performance
  • RWKV-TS: A time series task model based on RWKV, with low latency and memory usage

Robotics/Embodied Intelligence

  • DecisionRWKV: Experience replay (experience replay) + Decision-RWKV model, a lifelong learning algorithm suitable for robots.
  • OccRWKV: An efficient 3D semantic occupancy prediction with linear complexity

RWKV Tokenizers

Prompt Collections

  • Awesome-RWKV-Prompts: User-friendly and ready-to-use RWKV Prompts examples, suitable for all users.
  • (May be Outdated) RWKV_chains: Prompts that enable the RWKV model to work with Langchain

Other RWKV Projects