As an emerging trend in the area of edge computing, edge workloads tend to be managed and orchestrated by k8s. In the meantime, as the the top one workload of edge computing, edge AI accelerations have been enabled by different vendors' edge AI accelerators quickly, including Nvidia edge GPU series, Intel Movidius VPU, Google edge TPU, etc. Actually you can see many ASIC-based edge AI accelerators and even some high-end CPUs used in edge AI. Obviously, edge users have the challenges around empowering these heterogeneous edge AI on the edge with upstream k8s or those edge k8s versions due to missing a general unified framework on k8s. Here we'd like to introduce our unified framework as a plugin to k8s with the following key mechanisms - 1. Extend the Node Feature Discovery to detect edge AI accelerators automatically 2. Unify different vendors' device plugin to provision work nodes according to it's own edge AI accelerator 3. Introduce transparent backend accelerations to boost ML upstream frameworks such as Tensorflow, Pytorch, etc 4. Attaching remote GPU to edge In our project we provide a unified edge AI framework to help k8s empower heterogeneous AI acceleration on the edge.
Click here to view captioning/translation in the MeetingPlay platform!
