Run PixArt 3D AI on Just 6GB VRAM

Summary

PixArt 3D, a new open-source AI model for generating 3D models from 2D images, can now be run on as little as six gigabytes of VRAM, significantly lower than the previously required twenty-four gigabytes. Built upon the Trellis 2 model, PixArt 3D offers improved accuracy in matching the input image, particularly for the front view. This accessibility is achieved through GGUF quantization, made possible by contributions from the AI community. To get started, users need an Nvidia RTX series GPU or newer, and a specific installation guide for Trellis 2, which sets up necessary software like Python and CUDA. The process involves downloading a GGUF installation script and running it within ComfyUI. Users can then select different quantization levels, like Q4 for lower VRAM usage or Q8 for higher quality. Key steps include enabling the 'remove background' option in the TripoSR pre-process node. While PixArt 3D excels at shape accuracy, especially from the front, its texturing can be less refined, sometimes requiring separate texturing tools like TripoSR. Tips for optimal results include using clear, high-quality input images with solid backgrounds and ensuring no other applications are running to free up system resources. Compared to Trellis 2, PixArt 3D generally produces cleaner and more defined shapes, though Trellis 2 may offer more detail in certain complex scenes like buildings. The developers are actively working on improvements, including a multi-view feature.