Codex Now Controls Your Computer: What You Need to Know

Summary

Codex is no longer just a coding assistant; it can now interact with any application on your computer through its new "computer use" capability. This allows Codex to perform tasks within your local apps, making computer use a significant part of its functionality. The onboarding process is designed to be simple, requiring just a few clicks to grant permission for Codex to access specific applications. For example, Codex can now automatically create a new Mac virtual machine in the UTM app or play music on Spotify and set reminders in the Reminders app simultaneously. This capability allows Codex to work in the background without interrupting your own computer use. The system leverages multimodal models for visual understanding and can also use faster, non-multimodal models like Spark for superhuman speed in tasks like debugging. Future developments aim for computer use to operate at 2 to 10 times the speed of a human. Safety is addressed by requiring explicit user permission for each application Codex interacts with, ensuring it cannot access sensitive data without authorization. This "computer use" feature is currently available on Mac and will be rolling out to Windows soon, aiming to save users significant time and streamline complex workflows.