Because the 'berry often needs something to show things on. Because these displays are cheap(ish).
Display units are a twodimensional matrix of pixels. Some are "permanent" (eg. e-ink, where a pixel is set and stays set), others need dynamic refreshing (LCD, OLED, analog video).
The displays with refresh requirements need to be fed with steady stream of data, sequentially "painting" each pixel again in regular intervals.
The display controller can be smart, having its own "framebuffer" memory in which the image is stored and written to each pixel as the display panel timing requires. Then only the changed data have to be communicated to the controller. These are friendly to microcontrollers and other resource-limited applications.
Or it can be dumb, and rely on steady stream of well-timed data coming in. The framebuffer is still there, but it has to be maintained by the component that serves the data. A HDMI display, VGA monitor, or an analog TV are good examples here; the data come in as three (or six) serial digital streams with clock in fourth stream (HDMI), three analog channels with two digital horizontal/vertical synchronization pulses (VGA), or one composite signal with sync and brightness and color all mixed together. Or some intermediate flavors.
Several concepts appear here:
The image on screen is typically a "subset" of the "real" image going to the display. There are empty spaces before and after each line, and on top and bottom of each screen. In analog screens this was forced by the need to give the electron beam some time to return back to the line left or screen top. These empty spaces also house the synchronization pulses, the color synchronization bursts (analog trick for modulation of color carrier to the brightness signal, hack with backward compatibility with black/white screens), and in some cases additional data like eg. teletext.
In digital displays improper setting of these can manifest as part of the image on the sides "eaten", or by black bars on the sides.
Analog displays, typically CRTs, have a flat unstructured phosphor screen illuminated with a focused electron beam. The beam is deflected horizontally and vertically, typically by sending electrical current through a pair of magnetic coils. (Some tubes use high voltage on plates, for electrostatic deflection; these are usually found in oscilloscopes or other special applications. TVs and monitors overwhelmingly use magnetic deflection.)
With color displays, several kinds of fine structures are imposed on the screen, with triangular dots or parallel lines of red/green/blue phosphors and three separate electron guns aligned to hit just these.
The beam lights up the spot on the screen it is aimed to. The combination of horizontal and vertical deflection force determines the coordinates. Most typically, the beam is scanned over the screen sequentially, left to right, then line by line top to bottom.
(Other variant can be center-to-edge and gradually changing the angle along a circle, typical example is the radar screens. Another is with fixed horizontal sweep and arbitrary vertical deflection, oscilloscopes use this. Yet another uses arbitrary deflection for both, this regime is used in "vector displays" or with some oscilloscopes. If the beam intensity can be modulated, sending a fast sawtooth to horizontal deflection, slower sawtooth to vertical, and video signal to brightness can turn such oscilloscope into a black-and-white television. Such arbitrary-scanning-speed "TVs" are employed in scanning electron microscopes.)
Analog displays generate their own "clocks" for the electron beam sweep - a sawtooth signal with frequency matching the sync pulse frequency and phase synchronized to the edge of the sync pulse. "Pixel clock" is not used here, the color/image signal is analog. The image is written by an electron beam scanning over the phosphor screen, modulated by the analog data for "pixel" brightness.
The synchronization frequency can be fixed and determined by a standard (PAL/SECAM, NTSC, and all their flavors), or variable in several combinations determined by a standard (VGA, HDMI) or completely arbitrary. The frequency for the sawtooth generator for the beam deflection is then fixed (TVs) or inferred from the synchronization impulses. The pulses serve to sync the generators in phase; when not synced properly, a black bar appears on the screen, vertical (maybe slanted) or horizontal, stationary or scrolling slow or fast depending on the mismatch between the signal and own generators frequency, or the screen can be completely garbled.
The images can be sent all at once (noninterlaced), or interlaced, in halves (even lines on one frame, odd lines on the other). The latter trades some image quality and bandwidth/frequency for refresh speed. Older "standard" analog televisions employ this.
Various signal distortions can happen on the analog signal. Most common is noise. Another common is loss of higher frequency components, which manifests as artefacts along vertical lines on the screen; the lines have subtle "echoes", the lost higher freq components turn a square signal edge to a decaying oscillation.
Color components can be encoded by several different means:
Monochrome displays use only the luma component. It is better to feed them with signal without chroma. By getting rid of the chroma component, there are no artefacts caused by intermodulations and the image has better quality.
Synchronization pulses can also be encoded in several ways, some exploiting that they never occur within the visible part of the video signal:
The sync pulses come with their "blanking intervals" and associated substructures:
The width of blanking intervals in analog signals was enforced by the inductive nature of the CRT deflection coils, which needed some time to reverse the polarity of their magnetic field.
The image can be:
The signal contains fairly large portions of "empty" space, unused for the visible video. There is time segment with the horizontal pulse, part of which is used for the "color burst" for color oscillator phase synchronization. There is the time between last line of frame and first line of the next frame, the "vertical blanking interval" or "VBI" - many empty lines of video are present there. This space is commonly used for encoding various additional data - close captioning and teletext, ghost-cancelling reference signal, PAL+ extensions, and others.
The number of lines in the image and the frame rates are consequences of historical developments.
The even-odd nature of interlaced video requires analog frequency division, by an odd number. The line frequency had to be easily divisible odd multiple of the frame frequency. The frame frequency was originally dictated by the mains AC frequency, which was also used as an early substitute for vertical synchronization. This simultaneously eliminated flicker from AC-powered studio lamps and interference from mains power AC-to-DC power supply filtration and stray magnetic fields from transformers.
The framerate-to-mains-grid binding gives the main resolution between PAL/SECAM and NTSC formats, the first being based on 50 Hz (with 25fps framerate), the latter being based on 60 Hz (with 30 fps framerate).
The multiplication requirement gives the 525 scanlines for NTSC (3*5*5*7) and 625 lines for PAL/SECAM (5x5x5x5). Other older systems used similar multiples, 3x3x3x3x5 for British 405-line and 3x3x7x13 for French 819-line. [ref]
Some standards:
standard frames fields hfreq lines visible VBI NTSC 30 60 15.750 525 480 45 "480i60", "525i", "NTSC"; USA, since 1940 NTSC color 29.97 59.94 15.734 525 480 45 "480i60", "525i", "NTSC"; color NTSC, to avoid interference with sound carrier; 60/1.001; close enough to sync NTSC color 29.97 59.94 15.734 525 480 45 "240p60", used in some game emulators PAL/SECAM 25 50 15.625 625 576 49 "576i50", "625i", "PAL" 576p50 25 50 15.625 625 576 49 "288p50", used in some game emulators 405-line 25 50 10.125 405 pre-WW2, British standard, Ireland, Hong Kong; abandoned in UK in 1985, still kept alive by enthusiasts; monochrome-only 819-line 25 50 20.475 819 755 64 "755i" French "HD", Belgium, Luxembourgh, Monte Carlo, Morocco and other French Africa; abandoned in 1980s 441 25 50 11.025 441 pre-WW2, Germany, USSR 441 30 60 13.230 441 pre-WW2, USA 455 25 50 11.375 455 pre-WW2, France 567 25 50 14.175 567 1029 25 50 25.725 1029 WW2 Germany, 1940, for transmission of military maps; too wide bandwidth needed but exceeds quality of 16mm film 720p60 1080i60
NTSC uses 3.58 MHz sine wave color subcarrier, PAL uses 4.43 MHz sine wave.
Analog video signal can carry a host of other data:
Sync signal polarities can vary. In analog video it is always to the negative side, below the black level. When separate, they can be mostly H with pulses to L, or mostly L with pulses to H. The "filtered DC" component used to be employed to hint the early monitors what frequencies they should set.
Analog monitors have different needs for care and feeding.
------ levels ------ video sync type signals offset sync impedance polarity composite video 0..0.7 V AC-coupled within 75 ohm negative RGB SCART 0..0.7 V AC-coupled 75 ohm negative VGA 0..0.7 V yes 5V TTL 75 ohm configurable arcade monitors 2..5 V yes,1-2V 5V TTL 1-10 kΩ negative
Some video output circuits for 75 ohm, when unloaded, can give amplitude up to 1.4 volts.
Similar problematics is the analog VGA video. Three-row 15-pin D-sub connector, separate RGB analog signals, separate hsync vsync signals, optional I2C for an EEPROM with EDID data for the attached monitor's capabilities.
The timings are a wild mess: http://martin.hinner.info/vga/timing.html
Raspberry Pi composite video output supports multiple standards, chosen by sdtv_mode in config.txt:
The GPU setting details are handled in the firmware. Some firmwares allow more exotic settings:
Generally:
The aspect ratio of the output signal can be set by sdtv_aspect in config.txt:
The colorburst and color modulation can be disabled by sdtv_disable_colorburst=1 in config.txt.
Raspberry Pi 4 requires explicit enabling of the analog video output by enable_tvout=1 in config.txt; due to interrelationship between clock generators this mildly slows down the entire system.
The GPU firmware that is interpreting these directives is one of the start_something.elf files:
The supported directives can be figured out by running strings command on the .elf file, and eventually grepping the output.
The firmware under test on this machine has no support for composite_timings, and the SDTV directives found are:
The display size is set in the framebuffer commands in the config.txt file.
Some directives from the elf file are:
framebuffer_width and framebuffer_height are the visible resolution of the screen. "Real" resolution, raw, sent to the display, is this plus the overscans.
There is a firmware that allows explicit settings of the timing details:[ref]
composite_timings=h_active_pixels h_front_porch h_sync_width h_back_porch v_active_lines v_front_porch v_sync_width v_back_porch even_active_lines even_front_porch even_sync_width even_back_porch first_field_odd
NTSC: composite_timings=720 14 64 60 480 3 3 16 240 3 3 16 1 PAL: composite_timings=720 20 64 60 288 2 3 20 288 2 3 19 0 PAL60: composite_timings=720 20 64 60 240 3 3 16 240 4 3 16 1
In kernel, the videomode setting is a struct: https://01.org/linuxgraphics/gfx-docs/drm/API-struct-drm-display-mode.html
Setting the timings explicitly is being discussed in retrocasting and game console emulation communities.
https://www.raspberrypi.org/forums/viewtopic.php?t=271568
The resolutons are always fixed for standard PAL and NTSC:
Framebuffer of another size will be scaled to this by the GPU. This may lead to ugly artefacts.
From console, /opt/vc/bin/tvservice selects the HDMI or composite output and sets basic parameters.
The Raspberry Pi HDMI interface allows setting a wide range of resolutions and timings. There are many presets and if that is not enough, there is a mode with completely arbitrary timings.
Typical problem is when the raspi doesn't recognize the screen is attached. Here, in config.txt,
hdmi_force_hotplug=1
hdmi_drive=2
Sometimes the signal is too weak. Force stronger by
config_hdmi_boost=4
When nothing seems to work, try
hdmi_safe=1
hdmi_force_hotplug=1
hdmi_ignore_edid=0xa5000080
config_hdmi_boost=4
hdmi_group=2
hdmi_mode=4
disable_overscan=0
overscan_left=24
overscan_right=24
overscan_top=24
overscan_bottom=24
It is possible to control other devices over HDMI, if they support CEC (Consumer Electronics Control).
From raspi, CEC can be operated using cec-client (apt-get install cec-utils).
CEC signal is a separate wire on the HDMI cable. It is electrically identical to AV.link (also known as exTViewLink, SmartLink, Q-Link, EasyLink), with higher-level protocol. The interface is single-wire, pullup-to-3.3V open-collector, with synchronous timing-based communication. The timings are as follows:
start bit 4.5+-0.2ms L=3.7+-0.2ms bit 0 2.4+-0.35ms L=1.5+-0.2ms bit 1 2.4+-0.35ms L=0.6+-0.2ms observed 1.05+-0.2ms after falling edge ack pull bit to L within 0.35ms from falling edge, hold for nominal 0 timingReceiver can hold bit, turning transmitted 1 into 0. Transmitter has to listen during transmitting. This is used to ACK messages.
The bus is fairly slow, but tolerant to up to 7.3 nanofarads of capacitive loading. The speed is 416 bits per second, or about 41 bytes per second.
All devices share the same bus, all the CEC pins on connectors are connected together. (Therefore a faulty cable or driver on one takes down the entire network.) The connections are passive, so the data flow even through completely powered off devices.
Message consists of start bit, then sequence of 10 bits encoding 8bit bytes:
Each message starts with 4bit sender and recipient addresses. If two senders transmit at the same time, collision can be detected. If the address byte has EOM=1, it is a "ping" to see if the address is already claimed and alive. Otherwise the next byte is the opcode, followed with opcode-determined payload.
The device addresses are 4-bit. The address is assigned during power-on of the device when joining the network.
Addresses on the bus are ephemeral. A device first pings the address it wants to claim. If it responds with ACK, the device retries with another address.
Address 1111 (0x0F, 15) is used for sender when the sender did not decide on its address yet, and for recipient when the message is broadcast to all. Devices that do not need to receive non-broadcast messages can all have address 1111.
Example of message:
The Age of CRTs is fading. Phosphor-and-scanned-beam is gone, replaced with arrays of discrete pixels. Each pixel has fixed size and position. On color displays, each pixel has three subpixels, for red/green/blue channels. (Four, with yellow or white, can occur as well. Similar problematics as color masks on image sensors. Most typically, there are three, in vertical parallel lines.)
Ideally, the input signal has pixels corresponding to the display pixels. If this does not happen, various forms of downscaling or upscaling have to occur. This typically blurs the image, or makes it look pixelated.
The LCDs also "scan" the signal over their surface. Typically, the input RGB values are written to the subpixels with each pixel clock pulse, then the counter advances to the next pixel to the right. After each line is finished, next line is started.
With analog signals, the line-start signal is given explicitly from the horizontal pulse, and the pixel clock is reconstructed by the display by matching the clock generator frequency to produce just enough pixels to match its horizontal resolution for each line the signal source sends. The analog signal is then chopped at this frequency, converted to digital by an ADC, and fed to the digital circuitry.
With digital signals, the pixels often have explicit clock signal to mark the arrival of each of them. (Sometimes, for alleviation of eg. electromagnetic interference related problems, there is only one clock edge for several pixels. HDMI uses this.)
Parallel interfaces can use explicit clock signal, additional to the pixel data. Serial interfaces often use implicit clock, as bit clock is already present and pixel RGB data arrival can be inferred from arrival of 8, 16 or 24 bits of data over the serial line.
The video signal is usually a serial, or narrow parallel, analog or digital stream. Pixels (or "pixels", in analog signal case) are sent sequentially, with hints for beginnings of line and frame.
In composite video, the sync pulses, brightness, and color information are encoded together in one analog waveform. The brightness ("luma", "Y") varies between black and white levels (typically 0 to 0.7 volts), the syncs are going into negative, the color information ("chroma", "C") is modulated as higher-frequency signal onto the brightness signal. The color modulation is usually in the form of amplitude-phase, with reference signal to phase-lock the color decoder oscillator present in between the horizontal sync and the beginning of the visible video signal itself. Luma and chroma can interfere together, producing artefacts. This can be detrimental (unwanted colors on image edges) or beneficial (used by some early computers to produce colors).
Display panels are a type of digital screens. They can act as passive light valves that attenuate a backlight (LCD), or as active light sources (OLED). Feeding with signal is the same for both.
Displays can come as separate units with "outside world" connection. The most common are:
Such display units usually contain a regular directly-interfacing display and a board with signal converter.
"Bare" displays can also come with "chip-level" interfaces, designed to be directly interfaced with host electronics.
There are several kinds of interfaces for such displays:
Character LCD displays are a common and somewhat special case. These usually come with the HD44780 controller and 4 or 8 bit parallel bus.
With SPI, the signal is sent as a sequence of bytes. Activate the chipselect (CS) for the desired device on the bus, then pulse the clock (SCK) signal for each bit sent out on the output (MOSI, Master Out Slave In) and received from the input (MISO, Master In Slave Out).
This does not provide facility for determination between byte meanings, if they are raw signal data or some command for the display controller. Usually a control/data signalling wire has to be added. This signal is driven by the host computer and tells the display if we want to command it or if we just feed it with data to show.
Another additional signal is reset, RST. This initializes the display controller to a known state.
Some displays have also a dedicated signal to operate the backlight, to switch it on/off or even to control its brightness (usually by PWM).
The mandatory signals for a typical SPI display are:
Optional signals are:
Touchscreen displays with SPI interface require these signals:
Sometimes a full-fledged color graphical display is an overkill. Character LCDs, typically from 8x2 through 16x1 to 20x4, are common choice here. Most often they come with standard HD44780 controller.
In principle, these are similar to SPI displays.
The interfacing on the display itself is parallel:
In a simplified form, only 6 wires are needed:
To simplify the attachment and lower the number of wires, another bus interface is usually attached:
In some cases, especially with I2C, the CLK or E wire is driven separately, outside of the converter/shift register, as the single short pulse needed does not warrant the entire new byte being sent through the slow bus.
Unused bits can be employed for backlight control or keypad multiplexing.
The BCM chips support parallel display interface (DPI): https://www.raspberrypi.org/documentation/hardware/raspberrypi/dpi/README.md
The displays take:
On Raspberry Pi, the signals are assigned to BCM GPIO bank 0, mode 2:
alt2 --DPI output format-- function BCM hdr 2 3 4 5 6 7 alt0 alt1 alt3 alt4 alt5 note PCLK gpio0 27 565 565 565 666 666 888 I2C0 SDA SMI_SA5 AVEOUT_VCLK AVEIN_VCLK ID_SD; hat ID, I2C operated by GPU DE gpio1 28 I2C0 SCL SMI_SA4 AVEOUT_DSYNC AVEIN_DSYNC ID_SC; hat ID, I2C operated by GPU LCD_VSYNC gpio2 3 I2C1 SDA SMI_SA3 AVEOUT_VSYNC AVEIN_VSYNC LCD_HSYNC gpio3 5 I2C1 SCL SMI_SA2 AVEOUT_HSYNC AVEIN_HSYNC DPI_D00 gpio4 7 B3 B3 B2 B2 B0 GPCLK0 SMI_SA1 AVEOUT_VID0 AVEIN_VID0 JTAG_TDI DPI_D01 gpio5 29 B4 B4 B3 B3 B3 B1 GPCLK1 SMI_SA0 AVEOUT_VID1 AVEIN_VID1 JTAG_TDO DPI_D02 gpio6 31 B5 B5 B4 B4 B4 B2 GPCLK2 SMI_SOEN/SE AVEOUT_VID2 AVEIN_VID2 JTAG_RTCK DPI_D03 gpio7 26 B6 B6 B5 B5 B5 B3 SPI0 CS1 SMI_SWEN/SRWN AVEOUT_VID3 AVEIN_VID3 DPI_D04 gpio8 24 B7 B7 B6 B6 B6 B4 SPI0 CS0 SMI_SD0 AVEOUT_VID4 AVEIN_VID4 DPI_D05 gpio9 21 G2 B7 B7 B7 B5 SPI0 MISO SMI_SD1 AVEOUT_VID5 AVEIN_VID5 DPI_D06 gpio10 19 G3 G2 B6 SPI0 MOSI SMI_SD2 AVEOUT_VID6 AVEIN_VID6 DPI_D07 gpio11 23 G4 G3 B7 SPI0 SCK SMI_SD3 AVEOUT_VID7 AVEIN_VID7 DPI_D08 gpio12 32 G5 G2 G2 G4 G2 G0 PWM0 SMI_SD4 AVEOUT_VID8 AVEIN_VID8 JTAG_TMS DPI_D09 gpio13 33 G6 G3 G3 G5 G3 G1 PWM1 SMI_SD5 AVEOUT_VID9 AVEIN_VID9 JTAG_TCK DPI_D10 gpio14 8 G7 G4 G4 G6 G4 G2 UART0 Tx SMI_SD6 AVEOUT_VID10 AVEIN_VID10 UART1_Tx DPI_D11 gpio15 10 R3 G5 G5 G7 G5 G3 UART0 Rx SMI_SD7 AVEOUT_VID11 AVEIN_VID11 UART1_Rx DPI_D12 gpio16 36 R4 G6 G6 R2 G6 G4 FL0 SMI_SD8 UART0_CTS SPI1_CE2 UART1_CTS (SPI1 CS0?) DPI_D13 gpio17 11 R5 G7 G7 R3 G7 G5 FL1 SMI_SD9 UART0_RTS SPI1_CE1 UART1_RTS DPI_D14 gpio18 12 R6 R4 G6 PCM_CLK SMI_SD10 I2CSL_SDA/MOSI SPI1_CE0 PWM0 DPI_D15 gpio19 35 R7 R5 G7 PCM_FS SMI_SD11 I2CSL_SCK/SCK SPI1_MISO PWM1 DPI_D16 gpio20 38 R3 R6 R2 R0 PCM_DIN SMI_SD12 I2CSL_MISO SPI1_MOSI GPCLK0 DPI_D17 gpio21 40 R4 R3 R7 R3 R1 PCM_DOUT SMI_SD13 I2CSL_CE SPI1_SCK GPCLK1 DPI_D18 gpio22 15 R5 R4 R4 R2 SD0_CLK SMI_SD14 SD1_CLK JTAG_TRST DPI_D19 gpio23 16 R6 R5 R5 R3 SD0_CMD SMI_SD15 SD1_CMD JTAG_RTCK DPI_D20 gpio24 18 R7 R6 R6 R4 SD0_DAT0 SMI_SD16 SD1_DAT0 JTAG_TDO DPI_D21 gpio25 22 R7 R7 R5 SD0_DAT1 SMI_SD17 SD1_DAT1 JTAG_TCK DPI_D22 gpio26 37 R6 SD0_DAT2 TE0 SD1_DAT2 JTAG_TDI DPI_D23 gpio27 13 R7 SD0_DAT3 TE1 SD1_DAT3 JTAG_TMS
GPIO setting: [src]
dpi_output_format=value in config.txt: [ref]
23 16 15 8 7 0 (bits) - x x x - x x x - x x x - - x x x x x x x x x x f-f-f-f output format: 1=9bit 666, 2/3/4=16bit 565, 5/6=18bit 666, 7=24bit 888 o-o-o-o RGB order: 1=RGB, 2=BGR, 3=GRB, 4=BRG oe output enable mode: 0=DPI_OUTPUT_ENABLE_MODE_DATA_VALID, 1=DPI_OUTPUT_ENABLE_MODE_COMBINED_SYNCS ip invert pixel clock: 0=RGB data change on rising, stable on falling; 1=change on falling, stable on rising hsd hsync disable vsd vsync disable oed output_enable disable hsp hsync polarity: 0=HDMI default, 1=inverted vsp vsync polarity: 0=HDMI default, 1=inverted oep output_enable polarity 0=HDMI default, 1=inverted hph hsync phase: 0=DPI_PHASE_POSEDGE, 1=DPI_PHASE_NEGEDGE vph vsync phase: 0=DPI_PHASE_POSEDGE, 1=DPI_PHASE_NEGEDGE oeph output enable phase: 0=DPI_PHASE_POSEDGE, 1=DPI_PHASE_NEGEDGE
for custom timings, use:
dpi_group=2
dpi_mode=87
dpi_timings=<h_active_pixels> <h_sync_polarity> <h_front_porch> <h_sync_pulse> <h_back_porch> <v_active_lines> <v_sync_polarity> <v_front_porch> <v_sync_pulse> <v_back_porch> <v_sync_offset_a> <v_sync_offset_b> <pixel_rep> <frame_rate> <interlaced> <pixel_freq> <aspect_ratio>
overlays for DPI mode:
example config for VGA666 (three 6-bit resistive DACs):
resistors used:
example config for Adafruit 800x480 panel:
The parallel interface acts as up-to-24 bit wide synchronously clocked data bus, repeating ad nauseam the data array from the corresponding framebuffer.
The assignment of bits to RGB values is arbitrary. The same can be used for generation of any combination of analog and digital outputs, provided that the total is not more than 24.
If front/back porches and sync width can be set to zero, seamless "playback" of the signals could be achieved.
Arbitrary signals then can be generated, at tens-of-megahertzs samplerate, at precisely synchronized multiple channels.
Does not matter if the signals are analog or digital. This can be leveraged for generation of complex waveforms for e.g. clocking out data from CCD chips or clocking them in to some weird old LCDs.
The arbitrary waveforms can be used for driving weird kinds of displays. If two are used for deflection coils and third one for beam brightness, vector displays can be easily operated. Radar screens, with the polar-sweeped beam, can be driven by eg. writing the "sawtooths" with shifting phases to red/blue channels, with brightness to green. Some data lines can be also not connected to ADCs and instead used directly for driving other hardware (eg. shift registers).
TODO: try with an oscilloscope.
TODO: try with reclaimed displays from old electronics. Challenge: the bloody tiny connectors and lots and lots of wires.
TODO: some 'scope work and datasheet searches to find out how the signals look. How to identify signals by appearance, identify necessary timings and other settings. With digital oscilloscope with long enough memory, this can be automated.
The video interface in linux is often done via a framebuffer device - a block device with content that directly corresponds to the display output. It may be directly mapped video memory of a controller or a GPU, or can be periodically copied from there. Or can be periodically read and written to a hardware-interface bus.
The Raspi's default screen, whether configured for HDMI or for TV/analog, is /dev/fb0.
Let's skip, for now, the problematics of HDMI and MIPI/DSI.
SPI is the most commonly encountered display type attached via the GPIO bus.
The displays usually came as shields/hats, with the display chip controller signals already connected to fixed pins on the Raspi header. The SPI pins are given by the hardware SPI, but the assignment of CS, RST and RS (CD) signals can vary.
The controller chips also vary. This can bite the unwary, as attempt to talk to a chip in a dialect meant for another chip will lead to misunderstanding and usually no reaction at all. Maybe a mild "duh?" expression on the chip's epoxy face.
The SPI display is interfaced with using a kernel module. The usual architecture is fbtft, the generic "master" responsible for the additional /dev/fb? interface, sometimes fbtft_device that loads and configures the other modules, and fb_(chip) that's specific for the hardware display's controller flavor.
The display needs to have its mandatory signals attached properly, and without conflict with other devices. The communication is often unidirectional, without the MISO pin even used, therefore the computer has no idea if the display listens, understands, or even exists.
Better vendors list the pinout on the connector.
Sometimes a magnifying glass or resistance/diode tester is needed. Input diodes, present on most chips as ESD protection, are the friend here. Black wire on power supply positive, red wire on pins on the connector, a volt or so of a diode drop indicates something digital is connected. Serial resistors and pullups/pulldowns can confuse things a bit. Pin with no reaction at all may be unused and ignored in the trial-and-error.
SPI pins are given. CS pin is usually CS0 (CS1 is used by touchscreens). RST and RS (also named CD) varies between vendors and models, usually doesn't vary with model variants.
Displays can come as Raspi hats, mating directly onto the Raspi header, or they can be "loose" boards with a pin header that has to be wired to Raspi one by one. (Don't screw up the power wires. Messing up data/signals is unpleasant as things don't work, messing up power can easily break and kill things.)
Most SPI displays have the same or very similar wiring. However, the controllers for the LCD display itself vary greatly. Better vendors specify the chip used in documentation. Some vendors don't, and then it is a hide and seek (or trial and error) game; image search for the same board with the same appearance can unleash a different vendor with better documentation. Some documentations are wrong; this can be an especially unpleasant bite.
Sometimes a board has subtly different revisions, with similar appearance, same wiring, and different flavor of the chip that talks a different dialect. Getting the SPI to talk is getting the voice through - it is needed also to know what language has to be spoken.
An example is 3.2-inch display from Waveshare, with version "B" using ILI9340 and version "C" using ILI9341.
The controller chip is often bonded directly to the glass of the display panel. A sliver of silicon, attached to hundreds and hundreds of row and column lines. The configuration for the specific display attached can be baked right into the silicon, or has to be supplied by some form of an initialization sequence.
The display consists of a sandwich of polarization filters, glass slabs with lines and columns of electrodes and with transistors for each pixel, color masks, and liquid crystal fluid between the glass (with tiny glass spheres as spacers). The wiring from the rows/columns is attached to a controller chip, bonded to the glass. Typically a piece of flat ribbon cable goes out from that point, ready to be attached to an annoyingly tiny flat connector or broken/torn off, whatever the engineer in charge prefers.
To get the display operational, the kernel modules have to be loaded and properly configured.
Several approaches are possible:
The module loaded will rarely complain on load time if something is not grossly wrong. It however usually bitches on kernel log, accessible by dmesg command.
The module can load other dependent modules. This can be used to simplify the operation, and also can bite the unwary. lsmod is the command to list loaded modules, modprobe <module> to load one, rmmod <module> to unload one. Unloading must often be done in a certain sequence, as a module used by another module can't be unloaded until said another module is unloaded. Writing all rmmod commands in order to one commandline makes recalling the operation easier (eg. when testing parameters for a module).
A properly loaded framebuffer module will manifest as additional /dev/fbX device, usually /dev/fb1 (as the /dev/fb0 is the main HDMI screen). This doesn't guarantee proper communication between the hardware-specific module and the hardware, but it is a good start.
A way to write some data to the framebuffer is cat /dev/urandom > /dev/fb1 - this feeds the buffer with random noise (and "fails" by reaching the end of the device - which is normal and desired). A correctly attached display will then show fine-grained colorful noise, with each pixel having a random color. This is an expedient test when the display looks black and blank.
An example of a command to load the module for the display is:
modprobe fbtft_device custom name=fb_ili9341 gpios=reset:27,dc:22 speed=25000000 rotate=90 bgr=1
From kernel 5.4 the fbtft_device module is dropped and different syntax has to be used. https://github.com/notro/fbtft/wiki
Several modules will get loaded (only relevant ones are shown):
The syscopyarea, sysfillrect, sysimgblt and fb_sys_fops can be safely ignored now.
To retry the modules with different parameters, unload the three relevant ones in order - first fbtft, then fbtft_device and fb_ili3931 (or whatever hardware-specific one used):
rmmod fbtft; rmmod fbtft_device; rmmod fb_ili9341
Two Waveshare TFT LCD displays were acquired:
Both have 320x240 pixels color TFT SPI display, the same wiring of the display, the same touchscreen chip (ADS7846 or compatible), and three buttons connected to GPIOs. They differ in display controller; the B has ILI9340, the C has ILI9341.
The vendor-provided autoconfiguration software just loads a device tree overlay without further effort. This makes the touchscreen work on both, but only the version B worked out of the box. Version C did not work - the screen stayed white as after power-on, only with some flickering.
Manual approach had to be used.
To cause some confusion for newbies and annoyance for veterans, there are several ways to refer to the same pin:
The pairing has to be kept in mind - in hardware the header pin numbers are accessed, in software the signals are referred to by the corresponding chip pin numbers. "27" in signal then refers to physical pin 13. Confused already? Welcome to engineering. It will be fun, they said...
3.3V 1 + + 2 5V - BCM_2_SDA1 3 o + 4 5V - BCM_3_SCL1 5 o - 6 gnd - BCM_4_GCLK0 7 o o 8 BCM_14_Tx - gnd 9 - o 10 BCM_15_Rx - touchpanel IRQ TP_IRQ BCM_17 11 o o 12 BCM_18_PWM8 button1 button on GPIO reset RST BCM_27 13 o - 14 gnd LCD register sel. LCD_RS BCM_22 15 o o 16 BCM_23 button2 button on GPIO 3.3V 17 + o 18 BCM_24 button3 button on GPIO SPI to disp,tp SPI_MOSI BCM_10_MOSI 19 o - 20 gnd SPI from tp SPI_MISO BCM_9_MISO 21 o o 22 BCM_25 - (button 4 on 2.8") SPI clock SPI_CLK BCM_11_SCLK 23 o o 24 BCM_8_CE0 LCD_CS SPI LCD enable gnd 25 - o 26 BCM_7_CE1 TP_CS SPI touchpanel enable
The connector pin assignment is the same for both versions. The crucial pins are:
The SPI pins shared by both the touchscreen and the LCD are:
The pushbuttons are:
Boards can differ by the location of the RESET and DC, the presence, number and location of GPIO-attached buttons, the presence of touchscreen and its IRQ pin assignment. SPI pins are the same, CS0 is usually assigned for the LCD, CS1 for touchscreen or for SD card socket (if present).
On the 2.8" Waveshare display there is a fourth button, connected to BCM_25, header 22; the board is otherwise identical to the 3.2" boards.
The 3.5" Waveshare display has RESET on BCM_25 (header pin 22), DC on BCM_24 (header pin 18), and no touchscreen and pushbuttons. It also uses ILI9486 as the display controller.
board model controller res MISO MOSI SCK LCD_CS LCD_DC RESET LED TP-chip TP_CS TP_IRQ button-GPIOs Waveshare 3.2" B ILI9340 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM22/15 BCM27/13 ADS7846 BCM7/26 BCM17/11 18/12,23/16,24/18 Waveshare 3.2" C ILI9341 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM22/15 BCM27/13 ADS7846 BCM7/26 BCM17/11 18/12,23/16,24/18 waveshare35a/b/c ILI9486 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM25/22 ADS7846 BCM7/26 BCM17/11 waveshare4c ILI9486 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM25/22 ADS7846 BCM7/26 BCM17/11 piscreen ILI9486 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM25/22 BCM22/15 ADS7846 BCM7/26 BCM17/11 max 24 MHz SPI piscreen2r ILI9486 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM25/22 BCM22/15 ADS7846 BCM7/26 BCM17/11 max 64 MHz SPI pitft22 ILI9340 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM25/22 max 32 MHz SPI pitft28-capacitive ILI9340 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM25/22 FT6236 BCM7/26 BCM24/18 max 32 MHz SPI pitft28-resistive ILI9340 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM25/22 0x02/0 stmpe-ts BCM7/26 BCM24/18pu max 32 MHz SPI; uses gpio-backlight, stmpe-gpio pitft35-resistive HX8357D 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM25/22 0x02/0 stmpe-610 BCM7/26 BCM24/18pu max 32 MHz SPI; uses gpio-backlight, stmpe-gpio rpi-display ILI9341 320x240 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM23/16 BCM18/12 ADS7846 BCM7/26 BCM25/22pu max 32 MHz SPI mz61581 S6D02A1 128x160 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM25/22 BCM15/10 BCM18/12 ADS7846 BCM7/26 BCM4/7 max 128 MHz SPI tinylcd35 tinylcd 320x480 BCM9/21 BCM10/19 BCM11/23 BCM8/24 BCM24/18 BCM25/22 BCM18/12 ADS7846 BCM7/26 BCM5/29 4/7,17/11,24/18,25/22,27/13,pd max 48 MHz; keys 17=up,22=down,27=left,23=right,4=enter - overlay maps buttons to keyboard events; also PCF8563, DS1307 rpi-ft5406 capacitive touchscreen only FT5406 I2C vga666 [DPI] arbitrary BCM0=pixelclock,1=DE,2=vsync,3=hsync,4..27=DPI_D0..D23; color modes 565,666,888; uses DPI, parallel display interface, https://www.raspberrypi.org/documentation/hardware/raspberrypi/dpi/README.md
BCM9/21 means pin number 9 on the Broadcom chip, which is connected to pin 21 on the raspberry pi header. pu means pullup active. 0x02/0 with LED means gpio-backlight overriding to pin BCM2, with active-high (/1 is active-low); example of backlight-gpio section: BCM18/12 has PWM, handy for LED backlights
compatible = "gpio-backlight";
gpios = <0x2 0x2 0x0>;
default-on;
Module loading commands for the different boards are:
For 3.2" board rev B:
modprobe fbtft_device custom name=fb_ili9340 gpios=reset:27,dc:22 speed=25000000 rotate=90 bgr=1
modprobe fbtft_device custom name=fb_ili9341 gpios=reset:27,dc:22 speed=25000000 rotate=90 bgr=1
modprobe fbtft_device custom name=fb_ili9341 gpios=reset:25,dc:24 speed=25000000 rotate=90 bgr=1
The 2.8" board should be compatible with 3.2" model B (untested).
Common speeds are 16000000 (slow), 25000000, 32000000, and even higher (125000000 worked sometimes but was marginal).
The SPI clock speed together with the display resolution and colors determines the maximum framerate.
At 320x240, with 16 bit color per pixel (65536 colors, typ. 5+6+5 bits of RGB), the entire display takes (320*240*2=) 153,600 bytes. SPI at 32MHz can transfer 4,000,000 bytes per second. Top framerate for full refresh will be 26 fps. At 24bit color it will be a third less, some 17 fps. Overheads not counted.
Both boards use ADS7846 (or compatible/identical XPT2046) chip as the controller for the resistive touchscreen. The corresponding kernel module is ads7846.
When properly configured, it should create a HID event device at /dev/input/event0 (or event1 or so if other event devices are already present). Running cat /dev/input/event0 should show some garbage thrown on the screen at each touch of the display. If this happens, the hardware works.
The utility evtest (apt-get install evtest) shows the decoded events. Use evtest /dev/input/event0 to see the coordinates and touch pressure.
If the screen gives wrong coordinates, or rotates them against the display, something has to be reconfigured either with the kernel module or in the userspace software that takes the data.
So far the touchscreen was gotten to work using a device tree overlay.
Todo: manual module loading with parameters (how?) for debug.
To activate the pullup resistors on the pushbuttons, one way is using the gpio utility from WiringPi (apt-get install wiringpi):
gpio -g mode 18 up
gpio -g mode 23 up
gpio -g mode 24 up
Commmand gpio readall will show the status of all the pins. The modes for button pins have to be "IN", the values "1" when not pressed, "0" when pressed. The command shows the pin status at the moment of its invocation. To watch changes, use watch gpio readall which will show the output in two-second intervals.
There is also a gpiod package with more utilities. Many ways to do this.
Any sort of userspace queries can be used to attach function to button presses.
The config goes to /boot/cmdline.txt.
TODO: details
The more modern version of specifying hardware is in device tree overlays. These come in the /boot/overlays/ directory, usually with extension .dtbo, and are a binary blob of opaque mystery that makes some magic happen between the hardware and the kernel.
These are specified in the /boot/config.txt file, by using
dtoverlay=overlayname,parameter=value,parameter=value...
Each overlay and its associated parameters have one line.
Beware, some overlays can collide; the mix-and-match is not entirely free.
To the same file, add also line
dtdebug=on
vcdbg log msg
If the log is off, the command says
Otherwise a page of nice timestamped techspeak is produced. Problems can be then inferred much easier than from opaque nothingness. The perils of too much data vs too little data are open to discussion, but either is way better than no data at all.
If the changes in the file can't be saved, the /boot partition is most likely set to read-only to avoid corruption of the fickle FAT32 by dirty shutdowns. Use
mount -o remount,rw /boot
mount -o remount,ro /boot
The whole device tree, over which the overlays are applied, is stored in the file /boot/bcm<something>-rpi-<version>.dtb. Decompiling it yields a lot of nice text describing where are peripherals on GPIOs and registers, memory, clocks... With labels the overlay files refer to.
On power-on, the first thing to execute is the device bootloader. It resides in the firmware of the GPU. Bootloader mounts /boot partition (which at this point mounts as /mfs/sd), reads and parses the config.txt file, reads the appropriate bcm-something.dtb device tree file and applies overlays, reads the kernel binary and the cmdline.txt file with kernel parameters, and executes the kernel. From then, linux takes over.
(Bootloaders from the earliest version support the SD card. Later versions allow use of USB disk, and some allow network boot.)
The device tree and overlays contain data for the kernel about what modules to load with what configuration. Such hardware initialization can occur significantly earlier than if done from the startup scripts.
For the display/touchscreen to operate, the SPI functionality must be enabled. If it is not already so, add this to /boot/config.txt:
dtparam=spi=on
The .dtb and .dtbo files can be decompiled to text format and then recompiled back to binary using the dtc command (apt-get install device-tree-compiler). Running dtc /boot/overlays/something.dtbo produces some hierarchical obscure text that can be redirected to file, say something.dtc. After editing, dtc something.dtc > /boot/overlays/somethingNew.dtbo can produce a new, corrected file.
Sometimes the file can't be decompiled. (It's possible it also fails to load correctly.) Then a different version has to be sourced or something close-enough edited to fit.
The tree is exposed in the filesystem form in /proc/device-tree. It can be displayed in dts format by dtc -I fs /proc/device-tree.
The dtc command applied to the display overlay can provide useful data about the hardware.
The pins assigned are described by brcm,pins, brcm,function, and optional brcm,pull (for inputs). pins is a set of BCM pin numbers, function is 0x00 for input and 0x01 for output, pull is 0x00 for none, 0x01 for pulldown, 0x02 for pullup.
Example pieces from /boot/overlays/piscreen.dtbo, stripped from most of other data:
brcm,pins = <0x11 0x19 0x18 0x16>;
brcm,function = <0x0 0x1 0x1 0x1>;
piscreen@0 { // LCD panel
compatible = "ilitek,ili9486"; // uses ILI9486 chip, requests compatible driver
spi-max-frequency = <0x16e3600>; // max 24000000 Hz, or 24 MHz
rotate = <0x10e>; // rotate by 270 degrees if not said otherwise
bgr; // color order blue-green-red
fps = <0x1e>; // 30 fps
reset-gpios = <0xffffffff 0x19 0x0>; // reset pin 0x19 (BCM_25 aka pin 22)
dc-gpios = <0xffffffff 0x18 0x0>; // DC pin 0x18 (BCM_24 aka pin 18)
led-gpios = <0xffffffff 0x16 0x1>; // LED pin 0x16 (BCM_22 aka pin 15)
};
piscreen-ts@1 {
compatible = "ti,ads7846"; // uses ADS7846 chip, requests compatible driver
spi-max-frequency = <0x1e8480>; // max 2000000 Hz, or 2 MHz
interrupts = <0x11 0x2>; // interrupt on pin 0x11 (BCM_17 aka pin 11)
pendown-gpio = <0xffffffff 0x11 0x0>; // pen-down signal (touchscreen was touched) on pin 0x11
ti,swap-xy; // swap coordinates
ti,x-plate-ohms = [00 64];
ti,pressure-max = [00 ff];
};
The piscreen2r.dtbo file is similar. The pin assignment is the same, the spi-max-frequency is set to 0x3d09000, or 64 MHz, rotate is 0x5a or 90 degrees. The "init" block differs significantly, with piscreen2r being
init = <0x10000b0 0x0 0x1000011 0x20000ff 0x100003a 0x55 0x1000036 0x28 0x10000c0 0x11 0x9 0x10000c1 0x41 0x10000c5 0x0 0x0 0x0 0x0 0x10000b6 0x0 0x2 0x10000f7 0xa9 0x51 0x2c 0x2 0x10000be 0x0 0x4 0x10000e9 0x0 0x1000011 0x1000029>;
init = <0x10000b0 0x0 0x1000011 0x20000ff 0x100003a 0x55 0x1000036 0x28 0x10000c2 0x44 0x10000c5 0x0 0x0 0x0 0x0 0x10000e0 0xf 0x1f 0x1c 0xc 0xf 0x8 0x48 0x98 0x37 0xa 0x13 0x4 0x11 0xd 0x0 0x10000e1 0xf 0x32 0x2e 0xb 0xd 0x5 0x47 0x75 0x37 0x6 0x10 0x3 0x24 0x20 0x0 0x10000e2 0xf 0x32 0x2e 0xb 0xd 0x5 0x47 0x75 0x37 0x6 0x10 0x3 0x24 0x20 0x0 0x1000011 0x1000029>;
Given that the init sequences are "opaque" sequences of numbers without further explanation, adjusting these may be... somewhat challenging.
The [link?:.c|/boot/overlays/waveshare32b.dtbo] file decompiles as such:
It works out of the box, by putting waveshare32b.dtbo from the Waveshare.com link to /boot/overlays/, and adding
dtoverlay=waveshare32b
The file automatically adds support for both the display (showing as /dev/fb1) and the touchscreen (showing as /dev/input/event0, number may vary if other event sources are present).
On power-on, the display lights flat-white. A few seconds into boot the screen goes black and text of the console scrolls over it.
The file waveshare32c.dtbo from the Waveshare.com link did not work. The touchscreen stayed flat-white, with occasional mild barely visible flickering.
The touchscreen however worked correctly. But touchscreen without display is of... little use.
That meant actual thinking had to be done. Oy vey.
Running dtc on the file resulted in error:
Running strings on the file and seeing the pieces of text inside shown, inter alia,
This suggested the display may be based on TFT9341, an equivalent of ILI9341, and the reference to ILI9340 may be in error.
So the functional waveshare32b.dtbo was taken, decompiled to waveshare32b.dts, renamed to waveshare32cShad.dts, and edited, replacing the reference to ilitek,ili9340 with ilitek,ili9341, references to waveshare32b with waveshare32c (most likely just cosmetic anyway), and compiling to binary:
The warnings apparently aren't critical, as the overlay works.
On reboot, with the dtoverlay=waveshare32cShad, the display came to life.
Video playing with mplayer shown that the refresh rate is less than optimal, the image was tearing a bit on moving scenes.
The .dts file was edited again, the spi-max-frequency = <0xf42400> statement (16,000,000 Hz, or 16 MHz) was changed to spi-max-frequency = <0x3d09000>, nice 64,000,000, or 64 MHz, and compiled to .dtbo. After reboot, the video tearing almost vanished, leaving inconsequential remains.
On power-on, the display is blank, bright white.
After initialization by the kernel module, the display goes black.
To show the boot messages, this has to be added to /boot/cmdline.txt:
fbcon=map:10 fbcon=font:ProFont6x11 logo.nologo
Using /dev/fb1 as the framebuffer instead of the default /dev/fb0, for dealing with the attached SPI TFT LCD display.
Fills the fb1 framebuffer with random numbers; fails with write beyond the end of the device, which is expected and good. Should result in the entire display covered with random-colored pixels. Quick and dirty test if the screen is black.
cat /dev/fb1 > screenshot.bin
saves the display image to a file
cat screenshot.bin > /dev/fb1
fbset -i
shows info about the fb device
example for the Waveshare 3.2"(C) unit:
Frame buffer device information: Name : fb_ili9341 Address : 0 Size : 153600 Type : PACKED PIXELS Visual : TRUECOLOR XPanStep : 0 YPanStep : 0 YWrapStep : 0 LineLength : 640 Accelerator : No
cf. the HDMI display in default setting,
Frame buffer device information: Name : BCM2708 FB Address : 0x3eb64000 Size : 614400 Type : PACKED PIXELS Visual : TRUECOLOR XPanStep : 1 YPanStep : 1 YWrapStep : 0 LineLength : 1920 Accelerator : No
vlc
Plays video on the display. Very flexible, very powerful, sometimes annoying.
Refuses to run under root. vlc-wrapper does, but at first refuses with
vlc can be forced to run as root, by editing the binary, by eg. executing this:
TODO: get it to actually run
mplayer
Plays video on the display. Very very flexible. Actually works with framebuffers from console:
fbcp
Runs on background, copies and scales default screen (/dev/fb0) to /dev/fb1.
Uses calls to vc_dispmanx_... GPU API routines for hardware-accelerated scaling.
When this runs: catting urandom to /dev/fb1 has no effect, catting it to /dev/fb0 shows it on /dev/fb1; having this running on the backhround can cause some confusion.
A similar thing may be possible to use, for dramatic cost in speed, to process the display data eg. for some exotic kinds of displays, where one framebuffer with image gets converted to another framebuffer with heavily processed data more similar to digitally synthetized waveform than to a visible image.
fbi
shows images
fbi -T 1 -d /dev/fb1 -noverbose -a -t 5 <filename.jpg>
whitey
a console player of youtube content; https://www.oiepoie.nl/2015/02/18/retrocasting-with-a-raspberry-pi/
https://pypi.org/project/whitey/0.4/
On Raspberry Pi, the /dev/fb0 framebuffer is handled by the unit's GPU. This provides acceleration, OpenGL functionality, and many other goodies that fbtft-class framebuffers (eg. the /dev/fb1 from the added SPI display) don't have.