IMX477 camera board for motorized zoom lenses

With the introduction of motorized zoom lenses, the next logical step was matching the sensor board. Sure there are countless suitable sensors from various manufacturers like Sony semiconductors, Onsemi, Aptina, and many more. But lack of good support from the SOC side suggested starting from IMX477. It is a reasonably modern 12M image sensor used by many single-board computers like Raspberry Pi, NVIDIA Jetson, and others.

Raspberry Pi locks cameras with a security chip and prohibits non-authorized cameras with standard boards. Luckily this security feature is not used by compute modules. Motorized lenses have dedicated, and in most cases, unique direct mount features. So a custom camera module is a must, thus introducing the IMX477 MIPI CSI2 camera board.

This is the first of a few upcoming camera modules. Note multiple mounting features – this allows single-board use with many lenses.

Features of the lens kit:

  • Lens optical train – 3 stepper motors for Zoom, Focus and compensate lens groups
  • Iris
  • Two optical filters: IR CUT and NIR
  • Reference optical train elements
  • Runs on GRBL firmware ported on STM32 CPU with four axis motion planner

Using the camera with Raspberry Pi compute module

There is a great resource about the IMX477 camera on Raspberry Pi pages, feel free to learn how to control and use it from Raspberry Pi. Below is a simple recipe for streaming real-time video from RPI to a computer.

Run on computer first

gst-launch-1.0 udpsrc port=5004 ! "application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)JPEG,a-framerate=(string)40.000000,a-framesize=(string)1280-720,payload=(int)26" ! rtpjpegdepay ! decodebin ! autovideosink

Run on RPI as a second step

# Install libraries
sudo apt -y install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev

# Start streamer
raspivid -t 0 -cd MJPEG -awb greyworld -mm average -w 2028 -h 1520 -fps 30 -b 132000000 -o - | gst-launch-1.0 fdsrc ! "image/jpeg,framerate=30/1" ! jpegparse ! rtpjpegpay ! udpsink host=<COMPUTER_IP> port=5004

Control the lens

The SCE2-SDK control software can be forked from GitHub. Python code allows code reuse on many operating systems and keeps it human-readable, thus open for customizations.

pip install -r requirements.txt
  • Run demo with command
python main.py

Results

The lens is fairly complex – consists of 5 groups and 19 elements of optical components. Besides Focus and zoom actuators, it also has a distortion compensation element. Not all parameters and flange back distance are set perfectly, but 40x optical zoom looks sweet!

40x zoom – wide and tele pictures
VISIBLE vs NIR

Video on Youtube

And finally, demonstration video cycling through a few presets and changing filters.

Links

Comments ( 4 )

  1. ihor.kalinchuk
    Hello. I am trying to connect the camera to the Raspberry Pi 3. I did everything according to the instructions, but it does not work. Other regular cameras work fine. vcgencmd get_camera shows supported=1 detected=0 libcamera interfaces=0. Why might the camera not work?
    • saulius
      Hi, are you trying to connect to the full size Raspberry Pi 3 or compute module? Feel free to drop me an email (see contacts section on the web page) - will be easier to communicate.
  2. elango
    ihor.kalinchuk did you solved this issue..i'm getting same problem
    • saulius
      Hi, IMX477 sensor board works only with compute modules. Classic Raspberry Pi supports only cameras manufactured by Raspberry foundation.

Leave a reply

Your email address will not be published.