🚀 High-performance face detection, landmark detection, and emotion analysis powered by Rockchip NPU!
- ✨ Project Overview
- 🎯 Key Features
- 🏗️ Architecture
- 📱 Supported Devices
- 🛠️ Prerequisites
- 🚀 Getting Started
- 📁 Project Structure
- 💻 Usage Guide
- ⚙️ Configuration
- 🔧 Troubleshooting
- 📜 License
EVA-RKNN is a comprehensive Android application that demonstrates advanced computer vision capabilities on Rockchip's NPU-equipped devices. This project integrates three powerful AI models:
- 🎭 EVA (Emotion Valence Arousal) - Emotion analysis model
- 👤 Landmarks - Facial landmark detection (68 points)
- 🔍 RFB Quant - RetinaFace-based face detection
All models are optimized for Rockchip's Neural Processing Unit (NPU), delivering exceptional inference performance with minimal power consumption! 💪
| Feature | Description |
|---|---|
| ⚡ NPU Acceleration | Leverages Rockchip NPU for blazing-fast inference |
| 📷 Real-time Camera | Live camera preview with on-device AI processing |
| 🎭 Emotion Analysis | Valence and Arousal based emotion detection |
| 👤 Facial Landmarks | Precise 68-point facial landmark detection |
| 🔍 Face Detection | Robust face detection using RetinaFace |
| 🎨 Hardware Acceleration | RGA (2D Graphics Accelerator) for efficient image processing |
| 📱 Android Native | JNI bridge between Java and C++ for optimal performance |
EVA-RKNN/
├── 📱 Java Layer (UI & Camera)
│ ├── MainActivity - Entry point
│ ├── EvaActivity - Emotion analysis interface
│ ├── LandmarksActivity - Landmark detection interface
│ ├── RfbQuantActivity - Face detection interface
│ └── Wrapper Classes - JNI bridge
│
├── 🔧 C++ Layer (Inference Engine)
│ ├── EVA Module - Emotion analysis inference
│ ├── Landmarks Module - Facial landmark inference
│ ├── RFB Module - Face detection inference
│ ├── RGA Utils - Hardware-accelerated image processing
│ └── RKNN API - NPU interface
│
└── 🤖 AI Models (RKNN Format)
├── eva_quant.rknn - Quantized emotion analysis model
├── landmark_112_quant.rknn - Quantized landmark model
└── face_320_rk3588.rknn - Face detection model
This project is optimized for the following Rockchip SoCs:
- 🟢 RK3588 - Flagship octa-core with high-performance NPU
- 🟢 RK3568 - High-performance quad-core with NPU
- 🟢 RK3566 - Mid-range quad-core with NPU
- 🟢 RK3562 - Entry-level quad-core with NPU
-
Android Studio:
- 🪟 Windows:
2021.2.1or later - 🐧 Linux/macOS:
2021.3.1or later
- 🪟 Windows:
-
Android SDK: API Level 21+
-
NDK: For C++ compilation
-
CMake: Build system for native code
- Rockchip-based development board (RK3588/RK3568/RK3566/RK3562)
- Camera module (for real-time detection)
- RGA Driver Version: > 1.2.4
- librga Version: > 1.4
git clone https://github.com/your-repo/EVA-RKNN.git
cd EVA-RKNN- Launch Android Studio
- Select File > Open
- Navigate to the EVA-RKNN directory
- Wait for Gradle sync to complete
- Connect your Rockchip device via ADB
- Click Run > Run 'app' or press
Shift+F10 - Select your target device
- Wait for installation to complete
EVA-RKNN/
├── app/
│ ├── src/
│ │ ├── main/
│ │ │ ├── cpp/ # C++ native code
│ │ │ │ ├── eva/ # EVA emotion analysis module
│ │ │ │ │ ├── eva-rknn.cc # EVA inference implementation
│ │ │ │ │ ├── eva-rknn.h # EVA header file
│ │ │ │ │ └── eva-lib.cc # EVA library
│ │ │ │ ├── landmarks/ # Landmark detection module
│ │ │ │ │ ├── landmarks-rknn.cc # Landmarks inference implementation
│ │ │ │ │ ├── landmarks-rknn.h # Landmarks header file
│ │ │ │ │ └── landmarks-lib.cc # Landmarks library
│ │ │ │ ├── rfb/ # RFB face detection module
│ │ │ │ │ ├── rfb-quant-rknn.cc # RFB inference implementation
│ │ │ │ │ ├── rfb-quant-rknn.h # RFB header file
│ │ │ │ │ └── rfb-quant-lib.cc # RFB library
│ │ │ │ ├── rga/ # RGA hardware acceleration
│ │ │ │ │ ├── RockchipRga.h # RGA main API
│ │ │ │ │ ├── im2d.h # 2D image processing
│ │ │ │ │ └── ... # Other RGA utilities
│ │ │ │ └── rknn/ # RKNN API
│ │ │ │ └── rknn_api.h # RKNN runtime API
│ │ │ ├── java/com/huoshen/rknn/ # Java/Kotlin code
│ │ │ │ ├── wrapper/ # JNI wrapper classes
│ │ │ │ │ ├── EvaWrapper.java # EVA JNI wrapper
│ │ │ │ │ ├── LandmarksWrapper.java # Landmarks JNI wrapper
│ │ │ │ │ └── RfbQuantWrapper.java # RFB JNI wrapper
│ │ │ │ ├── CameraActivity.java # Camera base activity
│ │ │ │ ├── EvaActivity.java # EVA emotion analysis activity
│ │ │ │ ├── LandmarksActivity.java # Landmark detection activity
│ │ │ │ ├── RfbQuantActivity.java # Face detection activity
│ │ │ │ └── FlowActivity.java # Flow activity
│ │ │ ├── res/ # Resources
│ │ │ │ ├── raw/ # Model files & test images
│ │ │ │ │ ├── eva_quant.rknn # EVA quantized model
│ │ │ │ │ ├── landmark_112_quant.rknn # Landmark model
│ │ │ │ │ ├── face_320_rk3588.rknn # Face detection model
│ │ │ │ │ └── face*.jpg # Test images
│ │ │ │ ├── layout/ # UI layouts
│ │ │ │ └── values/ # Strings, colors, dimensions
│ │ │ └── AndroidManifest.xml # App manifest
│ │ └── CMakeLists.txt # CMake build configuration
│ └── build.gradle # App build configuration
├── gradle/ # Gradle wrapper
├── gradlew / gradlew.bat # Gradle execution scripts
├── LICENSE # License file
└── README.md / README_CN.md # Documentation
- Launch the app and select Face Detection
- Grant camera permissions when prompted
- Point the camera at faces
- View real-time detection results with bounding boxes and confidence scores
- Select Landmark Detection from the main menu
- The app will detect faces and overlay 68 facial landmarks
- Landmarks include eyes, nose, mouth, and facial contours
- Select Emotion Analysis (EVA)
- The app analyzes facial expressions and returns:
- Valence: Positive/negative emotion score
- Arousal: Activation/energy level
- Results are displayed in real-time!
If you encounter camera initialization issues:
- Locate the camera configuration file
- Modify the resolution parameters to match your camera's supported resolutions
- Common supported resolutions:
- 1920x1080 (Full HD)
- 1280x720 (HD)
- 640x360
- 320x180
To use your own RKNN models:
- Convert your model to RKNN format using RKNN Toolkit2
- Place the
.rknnfile inapp/src/main/res/raw/ - Update the model path in the corresponding wrapper class
- Rebuild and deploy
Error Message:
Execution failed for task ':app:mergeDebugNativeLibs'.
> 2 files found with path 'lib/arm64-v8a/librga.so' from inputs:
Solution:
Add the following to app/build.gradle:
android {
sourceSets {
main {
jniLibs.srcDirs = ['libs']
}
}
}For more details, see Android JNI Libs vs Imported Targets.
Error Message:
E/SurfaceView: Exception configuring surface
java.lang.RuntimeException: setParameters failed
Solution:
The camera doesn't support the requested resolution. Update the camera preview dimensions in the configuration file to use a supported resolution from the list printed in the logs.
Error: RGA-related errors or poor performance
Solution:
- Ensure RGA driver version > 1.2.4
- Ensure librga version > 1.4
- Update your Rockchip firmware to the latest version
This project is distributed under the license specified in the LICENSE file.
- Rockchip for providing the excellent NPU and RGA technology
- RKNN Model Zoo for model conversion references
- All contributors to this project
If you encounter any issues or have questions:
- Check the Troubleshooting section above
- Search existing issues on GitHub
- Open a new issue with detailed information
Made with ❤️ for Rockchip AI Development
🚀 Happy Coding!