Zynq UltraScale+ MPSoC
Soware Developer Guide
UG1137 (v2022.2) November 2, 2022
See all versions
of this document
Xilinx is creating an environment where employees, customers, and
partners feel welcome and included. To that end, we’re removing non-
inclusive language from our products and related collateral. We’ve
launched an internal initiative to remove language that could exclude
people or reinforce historical biases, including terms embedded in our
software and IPs. You may still find examples of non-inclusive
language in our older products as we work to make these changes and
align with evolving industry standards. Follow this link for more
information.
Table of Contents
Chapter 1: About This Guide.....................................................................................6
Introduction................................................................................................................................. 6
Intended Audience and Scope of this Document....................................................................7
Prerequisites................................................................................................................................ 7
Chapter 2: Programming View of Zynq UltraScale+ MPSoC
Devices............................................................................................................................. 9
Hardware Architecture Overview.............................................................................................. 9
Boot Process.............................................................................................................................. 12
Virtualization..............................................................................................................................15
System Level Reset Requirements.......................................................................................... 15
Security....................................................................................................................................... 16
Safety and Reliability.................................................................................................................19
Memory Overview for APU and RPU Executables.................................................................22
Chapter 3: Development Tools.............................................................................. 24
Vivado Design Suite.................................................................................................................. 24
Vitis Unified Software Platform............................................................................................... 26
Arm GNU Tools.......................................................................................................................... 28
Device Tree Generator..............................................................................................................29
PetaLinux Tools..........................................................................................................................29
Linux Software Development using Yocto............................................................................. 30
Chapter 4: Software Stack....................................................................................... 33
Bare Metal Software Stack....................................................................................................... 33
Linux Software Stack.................................................................................................................36
Third-Party Software Stack.......................................................................................................41
Chapter 5: Software Development Flow.......................................................... 42
Bare Metal Application Development.....................................................................................43
Application Development Using PetaLinux Tools................................................................. 45
Linux Application Development Using Vitis........................................................................... 45
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 2
Chapter 6: Software Design Paradigms........................................................... 50
Frameworks for Multiprocessor Development......................................................................50
Symmetric Multiprocessing (SMP).......................................................................................... 51
Asymmetric Multiprocessing (AMP)........................................................................................52
Chapter 7: System Boot and Configuration................................................... 56
Boot Process Overview.............................................................................................................56
Boot Flow....................................................................................................................................56
Boot Image Creation.................................................................................................................58
Boot Modes................................................................................................................................60
Detailed Boot Flow.................................................................................................................... 65
Disabling FPD in Boot Sequence............................................................................................. 68
Setting FSBL Compilation Flags...............................................................................................68
FSBL Build Process.................................................................................................................... 72
Using the Ethernet-Based Recovery Tool...............................................................................97
Chapter 8: Security Features..................................................................................99
Boot Time Security.................................................................................................................... 99
Bitstream Authentication Using External Memory............................................................. 110
Run-Time Security................................................................................................................... 113
Trusted Firmware-A................................................................................................................ 113
FPGA Manager Solution......................................................................................................... 116
Xilinx Memory Protection Unit...............................................................................................118
Xilinx Peripheral Protection Unit........................................................................................... 119
System Memory Management Unit......................................................................................119
A53 Memory Management Unit............................................................................................ 120
R5 Memory Protection Unit....................................................................................................120
TrustZone................................................................................................................................. 120
Chapter 9: Platform Management.................................................................... 121
Platform Management in PS..................................................................................................121
Wake Up Mechanisms............................................................................................................ 124
Platform Management for Memory......................................................................................125
DDR Controller.........................................................................................................................125
Platform Management for Interconnects............................................................................125
PMU Firmware......................................................................................................................... 126
Chapter 10: Platform Management Unit Firmware................................ 127
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 3
Features....................................................................................................................................127
PMU Firmware Architecture...................................................................................................128
Execution Flow.........................................................................................................................129
Handling Inter-Process Interrupts in PMU firmware......................................................... 131
PMU Firmware Modules.........................................................................................................135
Error Management (EM) Module.......................................................................................... 138
Power Management (PM) Module........................................................................................144
Scheduler..................................................................................................................................145
Safety Test Library...................................................................................................................145
CSU/PMU Register Access......................................................................................................146
Timers....................................................................................................................................... 147
Configuration Object.............................................................................................................. 150
PMU Firmware Loading Options........................................................................................... 152
PMU Firmware Usage............................................................................................................. 158
PMU Firmware Memory Layout and Footprint....................................................................164
Dependencies.......................................................................................................................... 166
Chapter 11: Power Management Framework.............................................167
Introduction............................................................................................................................. 167
Zynq UltraScale+ MPSoC Power Management Overview...................................................169
Power Management Framework Overview......................................................................... 173
Using the API for Power Management.................................................................................186
XilPM Implementation Details...............................................................................................193
Linux......................................................................................................................................... 196
Trusted Firmware-A (TF-A)..................................................................................................... 213
PMU Firmware......................................................................................................................... 216
Chapter 12: Reset........................................................................................................ 219
System-Level Reset................................................................................................................. 219
Block-Level Resets...................................................................................................................219
Application Processing Unit Reset........................................................................................ 220
Real Time Processing Unit Reset...........................................................................................221
Full Power Domain Reset....................................................................................................... 221
Warm Restart...........................................................................................................................221
Supported Use Cases..............................................................................................................225
Chapter 13: High-Speed Bus Interfaces......................................................... 248
USB 3.0......................................................................................................................................248
Gigabit Ethernet Controller....................................................................................................251
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 4
PCI Express...............................................................................................................................254
Chapter 14: Clock and Frequency Management....................................... 259
Changing the Peripheral Frequency.....................................................................................259
Chapter 15: Target Development Platforms................................................261
QEMU........................................................................................................................................261
Boards and Kits........................................................................................................................261
Chapter 16: Boot Image Creation...................................................................... 262
Appendix A: Libraries............................................................................................... 263
Appendix B: Additional Resources and Legal Notices........................... 264
Xilinx Resources.......................................................................................................................264
Documentation Navigator and Design Hubs...................................................................... 264
References................................................................................................................................264
Revision History.......................................................................................................................267
Please Read: Important Legal Notices................................................................................. 267
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 5
Chapter 1
About This Guide
Introduction
This document provides the soware-centric informaon required for designing and developing
system soware and applicaons for the Xilinx
®
Zynq
®
UltraScale+™ MPSoCs. The
Zynq UltraScale+ MPSoC family has dierent products, based upon the following system
features:
• Applicaon processing unit (APU):
○ Dual or Quad-core Arm
®
Cortex
®
-A53 MPCore
○ CPU frequency up to 1.5 GHz
• Real-me processing unit (RPU):
○ Dual-core Arm Cortex
®
-R5F MPCore
○ CPU frequency up to 600 MHz
• Graphics processing unit (GPU):
○ Arm Mali-400 MP2
○ GPU frequency up to 667 MHz
• Video codec unit (VCU):
○ Simultaneous Encode and Decode through separate cores
○ H.264 high prole level 5.2 (4Kx2K-60)
○ H.265 (HEVC) main, main10 prole, level 5.1, high Tier, up to 4Kx2K-60 rate
○ 8 and 10-bit encoding
○ 4:2:0 and 4:2:2 chroma sampling
For more details, see the Zynq UltraScale+ MPSoC Product Table and the Product Advantages.
Chapter 1: About This Guide
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 6
Intended Audience and Scope of this
Document
The purpose of this guide is to enable soware developers and system architects to become
familiar with:
• Xilinx soware development tools.
• Available programming opons.
• Xilinx soware components that include device drivers, middleware stacks, frameworks, and
example applicaons.
• Plaorm management unit rmware (PMU rmware), Trusted Firmware-A (TF-A), OpenAMP,
PetaLinux tools, Xen Hypervisor, and other tools developed for the Zynq UltraScale+ MPSoC
device.
Prerequisites
This document assumes that you are:
• Experienced with embedded soware development
• Familiar with Armv7 and Armv8 architecture
• Familiar with Xilinx development tools such as the Vivado
®
Integrated Design Environment
(IDE), the Vis™ unied soware plaorm, compilers, debuggers, and operang systems.
This document includes the following chapters:
• Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices: Briey explains the
architecture of the Zynq UltraScale+ MPSoC hardware. Xilinx recommends you to go through
and understand each feature of this chapter.
• Chapter 3: Development Tools: Provides a brief descripon about the Xilinx soware
development tools. This chapter helps you to understand all the available features in the
soware development tools. It is recommended for soware developers to go through this
chapter and understand the procedure involved in building and debugging soware
applicaons.
• Chapter 4: Soware Stack: Provides a descripon of various soware stacks such as bare
metal soware, RTOS-based soware and the full-edged Linux stack provided by Xilinx for
developing systems with the Zynq UltraScale+ MPSoC device.
• Chapter 5: Soware Development Flow: Walks you through the soware development
process. It also provides a brief descripon of the APIs and drivers supported in the Linux OS
and bare metal.
Chapter 1: About This Guide
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 7
• Chapter 6: Soware Design Paradigms: Helps you understand dierent approaches to develop
soware on the heterogeneous processing systems. Aer reading this chapter, you will have a
beer understanding of programming in dierent processor modes like symmetric mul-
processing (SMP), asymmetric mul-processing (AMP), virtualizaon, and a hybrid mode that
combines SMP and AMP.
• Chapter 7: System Boot and Conguraon: Describes the boong process using dierent
boong devices in both secure and non-secure modes.
• Chapter 8: Security Features: Describes the Zynq UltraScale+ MPSoC devices features you
can leverage to enhance security during applicaon boot- and run-me.
• Chapter 9: Plaorm Management: Describes the features available to manage power
consumpon, and how to control the various power modes using soware.
• Chapter 10: Plaorm Management Unit Firmware: Describes the features and funconality of
PMU rmware developed for Zynq UltraScale+ MPSoC device.
• Chapter 11: Power Management Framework: Describes the funconality of the Xilinx Power
Management Framework (PMF) that supports a exible power management control through
the plaorm management unit (PMU).
• Chapter 12: Reset: Explains the system and module-level resets.
• Chapter 13: High-Speed Bus Interfaces: Explains the conguraon ow of the high-speed
interface protocols.
• Chapter 14: Clock and Frequency Management: Briey explains the clock and frequency
management of peripherals in Zynq UltraScale+ MPSoC devices.
• Chapter 15: Target Development Plaorms: Explains about the dierent development
plaorms available for the Zynq UltraScale+ MPSoC device, such as quick emulators (QEMU),
and the Zynq UltraScale+ MPSoC boards and kits.
• Chapter 16: Boot Image Creaon: Describes Bootgen, a standalone tool for creang a
bootable image forZynq UltraScale+ MPSoC devices. Bootgen is included in the Vis soware
plaorm.
• Appendix A - Appendix K: Describe the available libraries and board support packages to help
you develop a soware plaorm.
• Appendix B: Addional Resources and Legal Noces: Provides links to addional informaon
that is cited throughout the document.
Chapter 1: About This Guide
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 8
Chapter 2
Programming View of Zynq
UltraScale+ MPSoC Devices
The Zynq
®
UltraScale+™ MPSoC supports a wide range of applicaons that require
heterogeneous mulprocessing. Heterogeneous mulprocessing system consists of mulple
single and mul-core processors of diering types. It supports the following features:
• Mulple levels of security
• Increased safety
• Advanced power management
• Superior processing, I/O, and memory bandwidth
• A design approach, based on heterogeneous mulprocessing presents design challenges,
which includes:
○ Meeng applicaon performance requirements within a specied power envelope
○ Opmizing memory access within heterogeneous mulprocessing system
○ Providing low-latency, coherent communicaons between various processing engines
○ Managing and opmizing system power consumpon in all operaonal modes
Xilinx
®
provides comprehensive tools for hardware and soware development on the
Zynq UltraScale+ MPSoC, and various soware modules such as operang systems,
heterogeneous system soware, and security management modules.
The Zynq UltraScale+ MPSoC is a heterogeneous device that includes the Arm
®
Cortex
®
-A53,
high-performance, energy-ecient, 64-bit applicaon processor, and also the 32-bit Arm
Cortex
®
-R5F dual-core real-me processor.
Hardware Architecture Overview
The Zynq UltraScale+ MPSoCs provide power savings, programmable acceleraon, I/O, and
memory bandwidth. These features are ideal for applicaons that require heterogeneous
mulprocessing.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 9
The following gure shows the Zynq UltraScale+ MPSoC architecture with next-generaon
programmable engines for security, safety, reliability, and scalability.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 10
Figure 1: Zynq UltraScale+ MPSoC Device Hardware Architecture
RPU
256 KB
OCM
LPD-DMA
CSU
PMU
Processing System
Cortex-R5F
32 KB I/D
128 KB TCM
Cortex-R5F
32 KB I/D
128 KB TCM
4 x 1GE
APU
Cortex-A53
32 KB I/D
Cortex-A53
32 KB I/D
Cortex-A53
32 KB I/D
Cortex-A53
32 KB I/D
GIC
SCU
ACP 1 MB L2
GPU
Mali-400 MP2
64 KB L2
2 x USB 3.0
NAND x8
ONFI 3.1
2 x SD3.0/
eMMC4.51
Quad-SPI
x 8
2 x SPI
2 x CAN
2 x I2C
2 x UART
GPIOs
SYSMON
MIO
Central
Switch
FPD-DMA
VCU
H.264/H.265
PCIe
Gen4
DisplayPort
v1.2 x1, x2
2 x SATA
v3.0
PCIe Gen2
x1, x2, or x4
SHA3
AES-GCM
RSA
Processor
System
BPU
DDRC (DDR4/3/3L, LPDDR3/4)
Programmabl
e Logic
128 KB RAM
PL_LPD
HP
GIC
LLLP
LLLP
RGMII
ULPI
PS-GTR
SMMU/CCI
GFC
USB 3.0
SGMII
Low Power Switch
To ACP
Low Power Full Power
Battery
Power
32-bit/64-bit
64-bit
M S
128-bit
M S
LPD_PL HPCHPM
GTY
Quad
GTH
Quad
Interlaken
100G
Ethernet
ACE
DisplayPort
Video and
Audio Interface
M => AXI Master S => AXI Slave
X23704-021320
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 11
The Zynq UltraScale+ MPSoC features are as follows:
• Cortex-R5F dual-core real-me processor unit (RPU)
• Arm Cortex-A53 64-bit quad/dual-core processor unit (APU)
• Mali-400 MP2 graphic processing unit (GPU)
• External memory interfaces: DDR4, LPDDR4, DDR3, DDR3L, LPDDR3, 2x Quad-SPI, and
NAND
• General connecvity: 2x USB 3.0, 2x SD/SDIO, 2x UART, 2x CAN 2.0B, 2x I2C, 2x SPI, 4x
1GE, and GPIO
• Security: Advanced Encrypon Standard (AES), RSA public key encrypon algorithm, and
Secure Hash Algorithm-3 (SHA-3)
• AMS system monitor: 10-bit, 1 MSPS ADC, temperature, voltage, and current monitor
• The processor subsystem (PS) has ve high-speed serial I/O (HSSIO) interfaces supporng the
protocols:
○ PCIe
®
: base specicaon, version 2.1 compliant, and Gen2x4
○ SATA 3.0
○ DisplayPort: Implements a DisplayPort source-only interface with video resoluon up to 4k
x 2k
○ USB 3.0: Compliant to USB 3.0 specicaon implemenng a 5 Gb/s line rate
○ Serial GMII: Supports a 1 Gb/s SGMII interface
• Plaorm Management Unit (PMU) for funcons that include power sequencing, safety,
security, and debug.
For more details, see the following secons of the Zynq UltraScale+ Device Technical Reference
Manual (UG1085): APU, RPU, PMU, GPU, and inter-processor interrupt (IPI).
Boot Process
The plaorm management unit (PMU) and conguraon security unit (CSU) manage and perform
the mul-staged boong process. You can boot the device in either secure or non-secure mode.
See Boot Process Overview or, see the Boot and Conguraon chapter of the Zynq UltraScale+
Device Technical Reference Manual (UG1085).
Boot Modes
You can use any of the following as the boot mode for boong from external devices:
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 12
• Quad SPI ash memory (QSPI24, QSPI32)
• eMMC18
• NAND
• Secure Digital Interface Memory (SD0, SD1)
• JTAG
• USB
The bootROM does not directly support boong from SATA, Ethernet, or PCI Express (PCIe). The
boot security does not rely on, and is largely orthogonal to TrustZone (TZ). The bootROM
(running on the Plaorm Management Unit) performs the security resources management (for
example, key management) and establishes root-of-trust. It authencates FSBL, locks boot
security resources, and transfers chain-of-trust control to FSBL (either on APU or RPU).
To understand more about the boot process in the dierent boot modes, see the ‘Boot and
Conguraon’ chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
QSPI24 and QSPI32
The QSPI boot mode supports the following:
• x1, x2, and x4 read modes for single Quad SPI ash memory (QSPI24) and x8 for dual QSPI
• Image search for MulBoot
• I/O mode is not supported in FSBL
Note: Single Quad-SPI memory (x1, x2 and x4) is the only boot mode that supports execute-in-place (XIP).
For addional informaon, see QSPI24 and QSPI32 Boot Modes.
eMMC18
The eMMC18 boot mode supports:
• FAT 16 and FAT 32 le systems for reading the boot images.
• Image search for MulBoot. The maximum number of searchable les as part of an image
search for MulBoot is 8,191.
For addional informaon, see eMMC18 Boot Mode.
NAND
The NAND boot supports the following:
• 8-bit widths for reading the boot images
• Image search for MulBoot
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 13
For addional informaon, see NAND Boot Mode.
SD
The SD boot supported version is 3.0. This version supports:
• FAT 16/32 le systems for reading the boot images.
• Image search for MulBoot. The maximum number of searchable les as part of an image
search for MulBoot is 8,191.
For addional informaon, see SD Boot Mode.
JTAG
You can download any soware images needed for the PS and hardware images needed for the
PL using JTAG.
IMPORTANT! In JTAG mode, you can boot the Zynq UltraScale+ MPSoC in non-secure mode only.
For addional informaon, see JTAG Boot Mode.
Zynq UltraScale+ devices do not support JTAG accesses while the CPU cores are powered down
randomly by the soware running on the device.
In case of PetaLinux, these kernel conguraon opons are known to be incompable with the
JTAG debugger:
• CONFIG_PERF_EVENTS
• CONFIG_FREEZER
• CONFIG_SUSPEND
• CONFIG_PM
• CONFIG_CPU_IDLE
USB
USB boot mode supports USB 3.0. It does not support MulBoot, image fallback, or XIP. It
supports both secure and non-secure boot mode. It is not supported for systems without DDR.
USB boot mode is disabled by default. For addional informaon, see USB Boot Mode.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 14
Virtualization
Virtualizaon allows mulple soware stacks to run simultaneously on the same processor,
which enhances the producvity of the Zynq UltraScale+ MPSoC. The role of virtualizaon varies
from system to system. For some designers, virtualizaon allows the processor to be kept fully
loaded at all mes, saving power and maximizing performance. For others systems, virtualizaon
provides the means to paron the various soware stacks for isolaon or redundancy.
For more informaon, see System Virtualizaon in the Zynq UltraScale+ Device Technical Reference
Manual (UG1085).
The support for virtualizaon applies only to an implementaon that includes Arm excepon
level-2 (EL2). Armv8 supports virtualizaon extension to achieve full virtualizaon with near
nave guest operang systems performance. There are three key hardware components for
virtualizaon:
• CPU virtualizaon
• Interrupt virtualizaon
• System MMU for I/O virtualizaon
System Level Reset Requirements
The system-level reset term is used to describe the system or subsystem level resets. ‘System’
reset (dierent from system-level resets) is a specic type of system-level reset. The following
table provides summary of system-level resets, which are described in details in subsequent
secons.
Table 1: System-Level Resets
Reset Type Description
External POR The external POR reset is triggered by external pin assertion. There are a
number of software only registers which are not reset by the POR resets. At first
POR boot, a safety system (requiring HFT1 by PS & PL) can be configured such
that a subsequent POR only resets PS (and not PL).
Internal POR Internal POR reset can be triggered by software register write, or by safety
errors. With the exception of error status register (which are reset by external
POR, but not by internal POR), internal POR resets the same thing as external
reset does. Internal-POR cannot be guaranteed without silicon validation (due
to in-rush power concern), so internal-POR is for internal purpose unless
validated.
System Reset
System reset is to be able to reset system excluding debug logic. To simplify
system reset, there are few other things (xBIST, scan clear, power gating) which
are not reset by this reset. Also, boot mode information is not reset by system
reset. The system reset can be triggered by external pin (SRST), or software
register write, or by safety errors.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 15
Table 1: System-Level Resets (cont'd)
Reset Type Description
PS Only Reset The PS only reset is to reset the PS while the PL remains active. This reset can be
triggered by hardware error signals or by software register write. This reset is a
subset of system reset (excluding the PL reset). If the PS reset is triggered by an
error signal, then the error is also transmitted to the PL.
FPD Reset The FPD reset resets all of the FPD power domain. It can be triggered by errors
or software register write. If the FPD reset is triggered by an error signal, then
the error is also transmitted to LPD & PL.
RPU Reset The RPU Reset is to reset the RPU in case of errors. While each of the R5 core
can be independently reset, but in lockstep, only R5_0 needs to be reset to reset
both the R5 cores. This reset can be triggered by errors or software register
write.
Security
The increasing ubiquity of Xilinx devices makes protecng the intellectual property (IP) within
them as important as protecng the data processed by the device. As security threats have
increased, the range of security threats or potenal weaknesses that must be considered to
deploy secure products has grown as well.
The Zynq UltraScale+ MPSoC provides the following features to help secure applicaons running
on the SoC:
• Encrypon and authencaon of boot images.
• Hardened crypto accelerators for use by the user applicaon.
• Secure methods of storing cryptographic keys.
Methods for detecng and responding to tamper events. See the Security chapter of the Zynq
UltraScale+ Device Technical Reference Manual (UG1085) for more informaon.
Configuration Security Unit
The following are some of the important responsibilies of the conguraon security unit (CSU):
• Secure boot.
• Tamper monitoring and response.
• Secure key storage and management.
• Cryptographic hardware acceleraon.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 16
The CSU comprises two main blocks as shown in the following gure. On the le is the secure
processor block that contains a triple redundant processor for controlling boot operaon. It also
contains an associated ROM, a small private RAM, and the necessary control/status registers
required to support all secure operaons. The block on the right is the crypto interface block
(CIB) and contains the AES-GCM, DMA, SHA, RSA, and PCAP interfaces.
Figure 2: Configuration and Security Unit Architecture
CSU PMU Switch
ROM
Validation
ROM
(128 KB)
RAM
(32 KB)
Triple
Redundant
MicroBlaze
SHA-3
384
AES-
GCM
256
Secure Stream Switch
PCAP
CSU DMA
CSU
Registers
Key
Management
To PL
Configuration
PMU ROM
Validation
To/From LPD Main Switch
Tamper
Sources
INTC
ECC
BBRAM
eFUSE
PUF
Operation
KUP
Family
CSU
Local
Registers
PUF
RSA
Multiplier
PSTP
Security Processor Block
Crypto Interface Block
X15318-032817
Aer
boot, the CSU provides tamper response monitoring. These crypto interfaces are available
during runme. To understand how to use these features, seethe XilFPGA Library v5.3 in the OS
and Libraries Document Collecon (UG643). See the Security chapter of the Zynq UltraScale+
Device Technical Reference Manual (UG1085) for more informaon.
• Secure Processor Block: The triple-redundant processor architecture enhances the CSU
operaons during single event upset (SEU) condions.
• Crypto Interface Block (CIB): Consists of AES-GCM, DMA, SHA-3/384, RSA, and PCAP
interfaces.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 17
• AES-GCM: The AES-GCM core has a 32-bit word-based data interface, with 256-bits of key
support.
• Key Management: To use the AES, a key must be loaded into the AES block. The key is
selected by CSU bootROM.
• SHA-3/384: The SHA-3/384 engine is used to calculate a hash value of the input image for
authencaon.
• RSA-4096 Accelerator: Facilitates RSA authencaon.
To understand boot image encrypon or authencaon, refer to the following:
• Chapter 7: System Boot and Conguraon
• Chapter 16: Boot Image Creaon
• The Security chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
• Boot and Conguraon informaon in the Zynq UltraScale+ Device Technical Reference Manual
(UG1085).
System-Level Protections
The system-level protecon mechanism involves the following areas:
• Zynq UltraScale+ MPSoC Linux soware stack relies on the Trusted Firmware-A (TF-A).
Protecon can be enhanced even further by conguring the XMPU and XPPU to provide the
system-level run-me security.
○ Protecon against buggy or malicious soware (erroneous soware) from corrupng
system memory or causing a system failure.
○ Protecon against incorrect programming, or malicious devices (erroneous hardware) from
corrupng system memory or causing a system failure.
○ Memory (DDR, OCM) and peripherals (peripheral control, SLCRs) are protected from illegal
accesses by erroneous soware or hardware to protect the system.
• The Xilinx memory protecon unit (XMPU) enforces memory paroning and TrustZone (TZ)
protecon for memory and FPD slaves. The XMPU can be congured to isolate a master or a
given set of masters to a developer-dened set of address ranges.
• The Xilinx peripheral protecon unit (XPPU) provides LPD peripheral isolaon and inter-
processor interrupt (IPI) protecon. The XPPU can be congured to permit one or more
masters to access an LPD peripheral. For more informaon, see the XPPU Protecon of Slaves
secon of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 18
Safety and Reliability
The Zynq UltraScale+ MPSoC architecture includes features that enhance the reliability of safety
crical applicaons to give users and designers increased condence in their systems. The key
features are as follows:
• Memory and cache error detecon and correcon
• RPU safety features
• System-wide safety features
To understand how to use these features, see Chapter 8: Security Features.
Safety Features
The Cortex-A53 MPCore processor supports cache protecon in the form of ECC on all RAM
instances in the processor using the following separate protecon elements:
• SCU-L2 cache protecon
• CPU cache protecon
These elements enable the Cortex-A53 MPCore processor to detect and correct a 1-bit error in
any RAM, and to detect 2-bit errors.
Cortex-A53 MPCore RAMs are protected against single-event-upset (SEU) such that the
processor system can detect and then, take specic acon to connue making progress without
data corrupon. Some RAMs have parity single-error detect (SED) capability, while others have
ECC single-error correct, double-error detect (SECDED) capability.
The RPU includes two major safety features:
• Lock-step operaon, shown in the following gure.
• Error checking and correcon, described further in Error Checking and Correcon.
Lock-Step Operation
Cortex-R5F processors support lock-step operaon mode, which operates both RPU CPU cores
as a redundant CPU conguraon called safety mode.
The Cortex-R5F processor set to operate in the lock-step conguraon exposes only one CPU
interface. Because Cortex-R5F processor only supports the stac split and lock conguraon,
switching between these modes is permied only while the processor group is held in power-
onreset (POR). The input signals SLCLAMP and SLSPLIT control the mode of the processor
group.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 19
These signals control the mulplex and clamp logic in the lock-step conguraon. When the
Cortex-R5F processors are in the lock-step mode (shown in the following gure), there must be
code in the reset handler to manage that the distributor within the GIC dispatches interrupts only
to CPU0. The RPU includes a dedicated interrupt controller for Cortex-R5F MPCore processors.
This Arm PL390 generic interrupt controller (GIC) is based on the GICv1 specicaon.
Figure 3: RPU Lock-Step Operation
X14824-062717
TCMs Associated
with CPU1
TCM A
TCM B
TCMs Associated
with CPU0
TCM A
TCM B
Shim Shim
Cortex-R5F
CPU0
Cortex-
R5F CPU0
Comparison and Synchronization Logic
Caches Associated
with CPU0
D-Cache
I-Cache
GIC
Tightly coupled memories (TCMs) are mapped in the local address space of each Cortex-R5F
processor; however, they are also mapped in the global address space where any master can
access them provided that the XPPU is congured to allow such accesses.
The following table lists the address maps from the RPU point of view:
Table 2: RPU Address Maps
Operation Mode Memory
R5_0 View (Start
Address)
R5_1 View (Start
Address)
Global Address
View (Start
Address)
Split Mode R5_0 ATCM (64 KB) 0x0000_0000 N/A 0xFFE0_0000
R5_0 BTCM (64 KB) 0x0002_0000 N/A 0xFFE2_0000
R5_0 instruction cache I-Cache N/A 0xFFE4_0000
R5_0 data cache D-Cache N/A 0xFFE5_0000
Split Mode R5_1 ATCM (64 KB) N/A 0x0000_0000 0xFFE9_0000
R5_1 BTCM (64 KB) N/A 0x0002_0000 0xFFEB_0000
R5_1 instruction cache I-Cache N/A 0xFFEC_0000
R5_1 data cache D-Cache N/A 0xFFED_0000
Lock-step Mode R5_0 ATCM (128 KB) 0x0000_0000 N/A 0xFFE0_0000
R5_0 BTCM (128 KB) 0x0002_0000 N/A 0xFFE2_0000
R5_0 instruction cache I-Cache N/A 0xFFE4_0000
R5_0 data cache D-Cache N/A 0xFFE5_0000
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 20
Error Checking and Correction
The Cortex-R5F processor supports error checking and correcon (ECC) schemes of data. The
data has similar properes although the size of the data chunk to which the ECC scheme applies
is dierent.
For each aligned data chunk, the processor computes and stores a number of redundant code
bits with the data. This enables the processor to detect up to two errors in the data chunk or its
code bits, and correct any single error in the data chunk or its associated code bits. This is also
referred to as a single-error correcon, double-error detecon (SEC-DED) ECC scheme.
System-Wide Safety Features
The system-wide safety features are designed to address error-free operaon of the
Zynq UltraScale+ MPSoC.
These features include the following:
Platform Management Unit
The plaorm management unit (PMU) in the Zynq UltraScale+ MPSoC executes the code loaded
from ROM and RAM within a at memory space, implements power safety rounes to prevent
tampering of PS voltage rails, performs logic built-in self-test (LBIST), and responds to a user-
driven power management sequence.
The PMU also includes some registers to control the funcons that are typically very crical to
the operaon and safety of the device. Some of the registers related to safety are as follows:
• GLOBAL_RESET: Contains reset for safety-related blocks.
• SAFETY_GATE: Gates hardware features from accidental enablement.
• SAFETY_CHK: Checks the integrity of the interconnect data lines by using target registers for
safety applicaons by periodically wring to and reading from these registers.
PMU Triple-Redundancy
The power management unit (PMU) contains triple-redundant MicroBlaze™ processors for a
high-level of system reliability and strong SEU resilience. PMU controls the power-up, reset, and
monitoring of resources within the enre system. The PMU performs mulple tasks including the
following tasks:
• Inializing the system during boot
• Managing power gang and retenon states for dierent power domains and islands
• Communicang the supply voltage sengs to the external power control devices
• Managing sleep states including the deep-sleep mode and processing of wake funcons
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 21
More details about PMU are available in Chapter 9: Plaorm Management.
Interrupts
The generic interrupt controller (GIC) handles interrupts. Both the APU and the RPU have a
separate dedicated GIC for interrupt handling. The RPU includes an Arm PL390 GIC, which is
based upon the GICv1 specicaon due to its exibility and protecon. The APU includes a
GICv2 controller. The GICv2 is a centralized resource for supporng and managing interrupts in
mul-processor systems. It aids the GIC virtualizaon extensions that support the
implementaon of the GIC in systems supporng processor virtualizaon.
The Zynq UltraScale+ MPSoC embeds an inter-processor interrupt (IPI) block that aids in
communicaon between the heterogeneous processors. Because PMUs can communicate with
dierent processors simultaneously, the PMU has four IPIs connected to the GIC of the PMU.
For more informaon on IPI roung to dierent processors, see the “Interrupts” chapter in the
Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Memory Overview for APU and RPU
Executables
The following tables give the congurable memory regions for APUs and RPUs.
Note:
•
In RPU lock-step mode (Lock-Step Operaon), R5_0_ATCM_MEM_0 and R5_0_BTCM_MEM_0 memory
address are mapped to R5_0_ATCM_LSTEP and R5_0_BTCM_LSTEP memory ranges respecvely in the
system address map.
•
In RPU split mode, R5_x_ATCM_MEM_0 and R5_x_BTCM_MEM_0 memory address are mapped to
R5_x_ATCM_SPLIT and R5_x_BTCM_SPLIT memory ranges respecvely in the system address map.
•
QSPI memory is accessible when QSPI controller is in linear mode.
See the System Addresses chapter of the Zynq UltraScale+ Device Technical Reference Manual
(UG1085) for more informaon.
See Real-me Processing Unit (RPU) and On-Chip Memory (OCM) secons of the Zynq UltraScale
+ Device Technical Reference Manual (UG1085) for more informaon on RPU, R5 and OCM.
Table 3:
Configurable Memory Regions for APUs
Memory Type Start Address Size
DDR Low 0x00000000 2 GB
DDR High 0x800000000 2 GB
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 22
Table 3: Configurable Memory Regions for APUs (cont'd)
Memory Type Start Address Size
OCM 0xFFFC0000 256 KB
QSPI 0xC0000000 512 MB
Table 4: Configurable Memory Regions for RPU Lock-Step Mode
Memory Type Start Address Size
DDR Low 0x100000 2047 MB
OCM 0xFFFC0000 256 KB
QSPI 0xC0000000 512 MB
R5_0_ATCM_MEM_0 0x00000 64 KB
R5_0_BTCM_MEM_0 0x20000 64 KB
R5_TCM_RAM_0_MEM 0x00000 256 KB
Table 5: Configurable Memory Regions for RPU Split Mode
Memory Type Start Address Size
R5_0
DDR Low 0x100000 2047 MB
OCM 0xFFFC0000 256 KB
QSPI 0xC0000000 512 MB
R5_0_ATCM_MEM_0 0x00000 64 KB
R5_0_BTCM_MEM_0 0x20000 64 KB
R5_1
DDR Low 0x100000 2047 MB
OCM 0xFFFC0000 256 KB
QSPI 0xC0000000 512 MB
R5_1_ATCM_MEM_0 0x00000 64 KB
R5_1_BTCM_MEM_0 0x20000 64 KB
Note: BootROM always copies First Stage Boot Loader (FSBL) from 0xFFFC0000 and it is not congurable.
If FSBL is compiled for a dierent load address, Bootgen may refuse it as CSU bootROM (CBR) does not
parse paron headers in the boot image but merely copies the FSBL code at a xed OCM memory
locaon (0xfffc0000). See Chapter 7: System Boot and Conguraon for more informaon on Bootgen.
Chapter 2: Programming View of Zynq UltraScale+ MPSoC Devices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 23
Chapter 3
Development Tools
This chapter focuses on Xilinx
®
tools and ows available for programming soware for
Zynq
®
UltraScale+™ MPSoCs. However, the concepts are generally applicable to third-party tools
as the Xilinx tools incorporate familiar components such as an
Eclipse-based integrated development environment (IDE) and the GNU compiler tool chain.
This chapter also provides a brief descripon about the open source tools available that you can
use for open source development on dierent processors of the Zynq UltraScale+ MPSoC.
A comprehensive set of tools for developing and debugging soware applicaons on
Zynq UltraScale+ MPSoC devices includes:
• Hardware IDE
• Soware IDEs
• Compiler toolchain
• Debug and trace tools
• Embedded OS and soware libraries
• Simulators (for example: QEMU)
• Models and virtual prototyping tools (for example: emulaon board plaorms)
Third-party tool soluons vary in the level of integraon and direct support for
Zynq UltraScale+ MPSoC devices.
The following secons provide a summary of the available Xilinx development tools.
Vivado Design Suite
The Xilinx Vivado
®
Design Suite contains tools that are encapsulated in the Vivado integrated
design environment (IDE). The IDE provides an intuive graphical user interface (GUI) with
powerful features.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 24
The Vivado Design Suite supersedes the Xilinx ISE soware with addional features for system-
on-a-chip development and high-level synthesis. It delivers a SoC-strength, IP- and system-
centric, next generaon development environment built exclusively by Xilinx to address the
producvity bolenecks in system-level integraon and implementaon.
All of the tools and tool opons in Vivado Design Suite are wrien in nave Tool Command
Language (Tcl) format, which enables use both in the Vivado IDE or the Vivado Design Suite Tcl
shell. Analysis and constraint assignment is enabled throughout the enre design process. For
example, you can run ming or power esmaons aer synthesis, placement, or roung. Because
the database is accessible through Tcl, changes to constraints, design conguraon, or tool
sengs happen in real me, oen without forcing re-implementaon.
The Vivado IDE uses a concept of opening designs in memory. Opening a design loads the design
netlist at that parcular stage of the design ow, assigns the constraints to the design, and then
applies the design to the target device. This provides the ability to visualize and interact with the
design at each design stage.
IMPORTANT! The Vivado IDE supports designs that target 7 series and newer devices only.
You can improve design performance and ease of use through the features delivered by the
Vivado Design Suite, including:
• The Processor Conguraon Wizard (PCW) within the IP integrator with graphical user
interfaces to let you create and modify the PS within the IP integrator block design.
VIDEO: For a beer understanding of the PCW, see the Quick Take Video: Vivado Processor
Conguraon Wizard Overview.
• Register transfer level (RTL) design in VHDL, Verilog, and SystemVerilog.
• Quick integraon and conguraon of IP cores from the Xilinx IP catalog to create block
designs through the Vivado IP integrator.
• Vivado synthesis.
• C-based sources in C, C++, and SystemC.
• Vivado implementaon for place and route.
• Vivado serial I/O and logic analyzer for debugging.
• Vivado power analysis.
• SDC-based Xilinx Design Constraints (XDC) for ming constraints entry.
• Stac ming analysis.
• Flexible oorplanning.
• Detailed placement and roung modicaon.
• Bitstream generaon.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 25
• Vivado Tcl Store, which you can use to add to and modify the capabilies in Vivado.
You can download the Vivado Design Suite from the Xilinx Vivado Design Suite – ML Edions.
Vitis Unified Software Platform
The Vis™ unied soware plaorm is an integrated development environment (IDE) for the
development of embedded soware applicaons targeted towards Xilinx embedded processors.
The Vis soware plaorm works with hardware designs created with Vivado Design Suite. The
Vis soware plaorm is based on the Eclipse open source standard and the features for
soware developers include:
• Feature-rich C/C++ code editor and compilaon environment
• Project management
• Applicaon build conguraon and automac Makele generaon
• Error navigaon
• Integrated environment for seamless debugging and proling of embedded targets
• Source code version control
• System-level performance analysis
• Focused special tools to congure FPGA
• Bootable image creaon
• Flash programming
• Script-based command-line tool
The Vis IDE lets you create soware applicaons using a unied set of Xilinx tools for the Arm
®
Cortex
®
-A53 and Cortex
®
-R5F processors as well as for Xilinx MicroBlaze™ processors. It
provides various methods to create applicaons, as follows:
• Bare metal and FreeRTOS applicaons for MicroBlaze
• Bare metal, Linux, and FreeRTOS applicaons for APU
• Bare metal and FreeRTOS applicaons for RPU
• User customizaon of PMU rmware
• Library examples are provided with the Vis tool (ready to load sources and build), as follows:
○ OpenCV
○ OpenAMP RPC
○ FreeRTOS “HelloWorld”
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 26
○ lwIP
○ Performance tests (Dhrystone, memory tests, peripheral tests)
○ RSA authencaon for prevenng tampering or modicaon of images and bitstream
○ First stage boot loader (FSBL) for APU or RPU.
You can export a block design, hardware design les, and bitstream les to the export directory
directly from the Vivado Project Navigator. For more informaon regarding the Vivado Design
Suite, see the Vivado Design Suite Documentaon.
All processes necessary to successfully complete this export process are run automacally. The
Vis IDE creates a new hardware plaorm project within the workspace containing the following
les:
• .project: Project le
• psu_init.tcl: PS inializaon script
• psu_init.c, psu_init.h: PS inializaon code
• psu_init.html: Register summary viewer
• system.hdf: Hardware denion le
The compiler can be switched as follows:
• 32-bit or 64-bit (applicaons that are targeted to Cortex-A53)
• 32-bit only (applicaons targeted to Cortex-R5F, and Xilinx MicroBlaze devices)
For the list of build procedures, see the Vis Unied Soware Plaorm Documentaon: Embedded
Soware Development (UG1400), where built-in help content lets you explore further aer you
launch the Vis IDE.
The Vis soware plaorm has the following IDE extensions.
• XSCT Console: Xilinx Soware Command-line Tool (XSCT) is an interacve and scriptable
command-line interface to the Vis soware plaorm. As with other Xilinx tools, the scripng
language for XSCT is based on Tools Command Language (Tcl). You can run XSCT commands
interacvely or script the commands for automaon. XSCT supports the following acons.
• Creang plaorm projects and applicaon projects
• Manage repositories
• Manage domain sengs and add libraries to domains
• Set toolchain preferences
• Congure and build applicaons
• Download and run applicaons on hardware targets
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 27
• Create and ash boot images by running Bootgen and program_ash tools
• Bootgen Ulity: Bootgen is a Xilinx tool that lets you stch binary les together and generate
device boot images. Bootgen denes mulple properes, aributes and parameters that are
input while creang boot images for use in a Xilinx device. Bootgen comes with both a
graphical user interface and a command line opon. The tool is integrated into the Vis
soware plaorm for generang basic boot images using a GUI, but the majority of Bootgen
opons are command line-driven. For more informaon on the Bootgen ulity, see the
Bootgen User Guide (UG1283).
• Program Flash: Program Flash is a tool used to program the ash memories in the design.
Various types of ash types are supported by the Vis soware plaorm for programming.
• Repositories: A soware repository is a directory where you can install third-party soware
components, as well as custom copies of drivers, libraries, and operang systems. When you
add a soware repository, the Vis soware plaorm automacally infers all the components
contained with the repository and makes them available for use in its environment. Your
workspace can point to mulple soware repositories.
• Program FPGA: You can use the Program FPGA feature to program FPGA using bitstream.
• Device Tree Generaon: Device tree (DT) is a data structure that describes hardware. This
describes hardware that is readable by an operang system like Linux so that it does not need
to hard code details of the machine. Linux uses the DT basically for plaorm idencaon,
runme conguraon like bootargs, and device node populaon.
For a detailed explanaon on the Vis IDE features, and to understand the embedded soware
design ow, see the Vis Unied Soware Plaorm Documentaon: Embedded Soware
Development (UG1400).
You can download the Vis tool from the Embedded Design Tools Download.
Arm GNU Tools
The Arm GNU open source toolchain is adopted for the Xilinx soware development plaorm.
The GNU tools for Linux hosts are available as part of Vis soware plaorm. This secon details
the open source GNU tools and Linux tools available for the processing clusters in the
Zynq UltraScale+ MPSoC.
The following table lists some of the Xilinx Arm GNU tools available for programming the APU,
RPU, and embedded MicroBlaze processors.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 28
Table 6: Xilinx Arm GNU Tools
Tool Description
aarch64-none-elf-gcc
aarch64-none-elf-g++
GNU C/C++ compiler.
aarch64-none-elf-as GNU assembler.
aarch64-none-elf-ld GNU linker.
aarch64-none-elf-ar A utility for creating, modifying, and extracting from
archives.
aarch64-none-elf-objcopy Copies and translates object files.
aarch64-none-elf-objdump Displays information from object files.
aarch64-none-elf-size Lists the section sizes of an object or archive file.
aarch64-none-elf-gprof Displays profiling information.
aarch64-none-elf-gdb The GNU debugger.
Device Tree Generator
The device tree (DT) data structure consists of nodes with properes that describe a hardware.
The Linux kernel uses the device tree to support a wide range of hardware conguraons.
In FPGAs, it is possible to have dierent combinaons of peripheral logics, each using a dierent
conguraon. For all the dierent combinaons, the device tree generator (DTG) generates
the .dts/.dtsi device tree les.
The following is a list of the .dts/.dtsi les generated by the device tree generator:
• pl.dtsi: Contains all the memory mapped peripheral logic (PL) IPs.
• pcw.dtsi: Contains the dynamic properes for the PS IPs.
• system-top.dts: Contains the memory, boot arguments, and command line parameters.
• zynqmp.dtsi: Contains all the PS specic and the CPU informaon.
• zynqmp-clk-ccf.dtsi: Contains all the clock informaon for the PS peripheral IPs.
For more informaon, see the Build Device Tree Blob page on the Xilinx Wiki.
PetaLinux Tools
The PetaLinux tools oer everything necessary to customize, build, and deploy open source
Linux soware to devices.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 29
PetaLinux tools include the following:
• Build tools such as GNU, petalinux-build, and make to build the kernel images and the
applicaon soware.
• Debug tools such as GDB, petalinux-boot, and oprofile for proling.
The following table shows the supported PetaLinux tools.
Table 7: PetaLinux Supported Tools
Tools Description
GNU Arm GNU tools.
petalinux-build Used to build software image files.
Make Make build for compiling the applications.
GDB GDB tools for debugging.
petalinux-boot Used to boot Linux.
QEMU Emulator platform for the Zynq UltraScale+ MPSoC device.
OProfile Used for profiling.
See the following documentaon for more details:
• PetaLinux Tools documentaon
• Zynq UltraScale+ MPSoC: Embedded Design Tutorial (UG1209)
• Libmetal and OpenAMP for Zynq Devices User Guide (UG1186)
Linux Software Development using Yocto
Xilinx oers the meta-xilinx Yocto/OpenEmbedded recipes to enable those customers with
in-house Yocto build systems to congure, build, and deploy Linux for Zynq
®
UltraScale+™
MPSoCs.
The meta-xilinx layer also provides a number of BSPs for common boards which use Xilinx
devices.
The meta-xilinx layer provides addional support for Yocto/OE, adding recipes for various
components. See meta-xilinx for more informaon.
You can develop Linux soware on Cortex-A53 using open source Linux tools. This secon
explains the Linux Yocto tools and its project development environment.
The following table lists the Yocto tools.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 30
Table 8: Yocto Tools
Tool Type Name Description
Yocto build tools Bitbake Generic task execution engine that
allows shell and Python tasks to be run
efficiently, and in parallel, while
working within complex inter-task
dependency constraints.
Yocto profile and trace tools
Perf Profiling and tracing tool that comes
bundled with the Linux Kernel.
Ftrace Refers to the ftrace function tracer but
encompasses a number of related
tracers along with the infrastructure
used by all the related tracers.
Oprofile System-wide profiler that runs on the
target system as a command-line
application.
Sysprof System-wide profiler that consists of a
single window with three panes, and
buttons, which allow you to start, stop,
and view the profile from one place.
Blktrace A tool for tracing and reporting low-
level disk I/O.
Yocto Project Development Environment
Developers can congure the Yocto project development environment to support developing
Linux soware for Zynq UltraScale+ MPSoCs through Yocto recipes provided from the Xilinx GIT
server. You can use components from the Yocto project to design, develop, and build a Linux-
based soware stack.
The following gure shows the complete Yocto project development environment. The Yocto
project has wide range of tools which can be congured to download the latest Xilinx kernel and
build with some enhancements made locally in the form of local projects.
You can also change the build and hardware conguraon through BSP.
Yocto combines a compiler and other tools to build and test images. Aer the images pass the
quality tests and package feeds required for SDK generaon are received, the Yocto tool
launches the Vis IDE for applicaon development.
The important features of the Yocto project are, as follows:
• Provides a recent Linux kernel along with a set of system commands and libraries suitable for
the embedded environment.
• Makes available system components such as X11, GTK+, Qt, Cluer, and SDL (among others)
so you can create a rich user experience on devices that have display hardware. For devices
that do not have a display or where you wish to use alternave UI frameworks, these
components need not be installed.
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 31
• Creates a focused and stable core compable with the OpenEmbedded project with which
you can easily and reliably build and develop Linux soware.
• Supports a wide range of hardware and device emulaon through the quick emulator (QEMU).
See the Xilinx Quick Emulator User Guide: QEMU for more informaon.
IMPORTANT! Enabling full Yocto of Xilinx QEMU is not available.
Figure 4: Yocto Project Development Environment
User Configuration
Metadata
(.bb+patches)
Machine(BSP)
Configuration
Policy Configuration
Source
Fetching
Patch
Application
Configuration /
Compile /
Autoreconf as
needed
Output
Analysis for
package
splitting plus
Package
relationships
.rpm
Generation
.deb
Generation
.ipk
Generation
QA
Tests
image
Generation
Images
Application
Development
SDK
Package Feeds
Source Mirror(s)
Upstream
Project
Releases
Local
Projects
SCMs
(optional)
Upstram Source
Metadata/Inputs
Build System
Output Packages
Process steps (Tasks)
Output Image Data
SDK
Generation
X14841-021317
You can download the Yocto tools and the Yocto project development environment from the
Yocto Project Organizaon.
For more informaon about Xilinx-provided Yocto features, see Yocto Features in the PetaLinux
Tools Documentaon: Reference Guide (UG1144).
Chapter 3: Development Tools
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 32
Chapter 4
Software Stack
This chapter provides an overview of the various soware stacks available for the Zynq
®
UltraScale+™ MPSoC devices.
For more informaon about the various soware development tools used with this device, see
Chapter 3: Development Tools. For more informaon about bare metal and Linux soware
applicaon development, see Chapter 5: Soware Development Flow.
Bare Metal Software Stack
Xilinx
®
provides a bare metal soware stack called the standalone board support package (BSP)
as part of the Vis™ soware plaorm. The Standalone BSP gives you a simple, single-threaded
environment that provides basic features such as standard input/output and access to processor
hardware features. The BSP and included libraries are congurable to provide the necessary
funconality with the least overhead. You can locate the standalone drivers at the following path:
<Xilinx Installation Directory>\Vitis\<version>\data\embeddedsw
\XilinxProcessorIPLib\drivers
You can locate libraries at the following path:
<Xilinx Installation Directory>\Vitis\<version>\data\embeddedsw\lib
\sw_services
The following gure illustrates the bare metal soware stack in the APU.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 33
Figure 5: Bare-Metal Software Development Stack
User Applications
Zynq UltraScale+ MPSoC Hardware
lwIP 211
XilFlash
XilSecure
XilFFS
Xilpm
XilSkey
Display
Driver
ZDMA
drivers
Ethernet
Driver
USB
Driver
SD card
Driver
Flash
Drivers
SPI, I2C,
UART
Drivers
SYSMON
Drivers
Libraries
Standalone
Drivers
XilFPGA
X17169-062121
Note: The soware stack of libraries and drivers layer for bare metal in RPU is same as that of APU.
The key components of this bare metal stack are:
• Soware drivers for peripherals including core rounes needed for using the Arm
®
Cortex
®
-
A53, Arm
®
Cortex
®
-R5F processors in the PS as well as the Xilinx
®
MicroBlaze™ processors in
the PL.
• Bare metal drivers for PS peripherals and oponal PL peripherals.
• Standard C libraries: libc and libm, based upon the open source Newlib library, ported to the
Arm Cortex-A53, Arm Cortex-R5F, and the MicroBlaze processors.
• Addional middleware libraries that provide networking, le system, and encrypon support.
• Applicaon examples including the rst stage boot loader (FSBL) and test applicaons.
The C Standard Library (libc)
libc library contains standard funcons that all C programs can use. The following table lists the
libc modules:
Table 9: Libc.a Functions and Descriptions
Header File Description
alloca.h Allocates space in the stack
assert.h Diagnostics code
ctype.h Character operations
errno.h System errors
inttypes.h Integer type conversions
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 34
Table 9: Libc.a Functions and Descriptions (cont'd)
Header File Description
math.h Mathematics
setjmp.h Non-local goto code
stdint.h Standard integer types
stdio.h Standard I/O facilities
stdlib.h General utilities functions
time.h Time function
The C Standard Library Mathematical Functions
(libm)
The following table lists the libm mathemacal C modules:
Table 10: libm.a Function Types and Function Listing
Function Type Supported Functions
Algebraic cbrt, hypot, sqrt
Elementary transcendental asin, acos, atan, atan2, asinh, acosh, atanh, exp, expm1, pow, log,
log1p, log10, sin, cos, tan, sinh, cosh, tanh
Higher transcendentals j0, j1, jn, y0, y1, yn, erf, erfc, gamma, lgamma, and gamma_ramma_r
Integral rounding eil, floor, rint
IEEE standard recommended copysign, fmod, ilogb, nextafter, remainder, scalbn, and fabs
IEEE classification isnan
Floating point logb, scalb, significand
User-defined error handling routine matherr
Standalone BSP
The libraries available with the standalone BSP are as follows:
• XilFatFS: A LibXil FATFile system and provides read/write access to les stored on a Xilinx
system ACE compact ash.
• XilFFS: Generic Fat File System Library.
• XilFlash: Xilinx ash library for Intel/AMD CFI compliant parallel ash.
• XilSecure: Xilinx Secure library provides an interface to access secure hardware (AES, RSA and
SHA) engines.
• XilSkey: Xilinx secure key library.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 35
• XilFPGA: A library that provides an interface to the Linux or bare-metal users for conguring
the programmable logic (PL) over PCAP from PS.
• XilPM: Xilinx Power Management (XilPM) provides Embedded Energy Management Interface
(EEMI) APIs for power management on Zynq UltraScale+ MPSoC.
• XilMailbox: The XilMailbox library provides the top-level hooks for sending or receiving an
inter-processor interrupt (IPI) message using the Zynq UltraScale+ MPSoC IPI hardware
• lwIP Library: An open source TCP/IP protocol suite that provides access to the core lwIP stack
and BSD (Berkeley Soware Distribuon) sockets style interface to the stack.
These libraries are documented in The C Standard Library (libc).
.
Linux Software Stack
The Linux OS supports the Zynq UltraScale+ MPSoC. With the sole excepon of the Arm GPU,
Xilinx provides open source drivers for all peripherals in the PS as well as key peripherals in the
PL. The following gure illustrates the full soware stack in APU, including Linux and an oponal
hypervisor.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 36
Figure 6: Linux Software Development Stack
Trusted
App1
Third Party Secure OS
ARM Trusted Firmware
EL0
EL1
EL2
EL3
Secure World
App1
App2 App3
Linux SMP
U-Boot/Hypervisor
Non-secure World
PMU Firmware
X18968-062121
The Armv8 excepon model denes excepon levels EL0–EL3, where:
• EL0 has the lowest soware execuon privilege. Execuon at EL0 is called unprivileged
execuon.
• Increased excepon levels, from 1 to 3, indicate an increased soware execuon privilege.
• EL1 runs the non-secure operang system in the non-secure world or a secure operang
system in the secure world when using a TEE architecture.
• EL2 provides support for processor virtualizaon. You may oponally include an open source
or commercial hypervisor in the soware stack.
• EL3 provides support for secure monitor soware. The Cortex-A53 MPCore processor
implements all the excepon levels (EL0-EL3) and supports both execuon states (AArch64
and AArch32) at each excepon level.
You can leverage the Linux soware stack for the Zynq UltraScale+ MPSoC in mulple ways. The
following are some of your opons:
• PetaLinux Tools: The PetaLinux tools include a branch of the Linux source tree, U-Boot as well
as Yocto-based tools to make it easy to build complete Linux images including the kernel, the
root le system, device tree, and applicaons for Xilinx devices. See the PetaLinux Product
Page for more informaon. The PetaLinux tools work with the same open source Linux
components described immediately below.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 37
• Open Source Linux and U-Boot: The Linux Kernel sources including drivers, board
conguraons, and U-Boot updates for the Zynq UltraScale+ MPSoC are available from the
Xilinx GitHub link, and on a connuing basis from the main Linux kernel and U-Boot trees as
well. Yocto board support packages are also available from the main Yocto tree.
• Commercial Linux Distribuons: Some commercial distribuons also include support for Xilinx
UltraScale+ MPSoC devices and they include advanced tools for Linux conguraon,
opmizaon, and debug. You can nd more informaon about these from the Xilinx
Embedded Compung page.
Multimedia Stack Overview
This secon describes the mulmedia soware stack in the Zynq UltraScale+ MPSoC.
The GPU and a high performance DisplayPort accelerate the graphics applicaon. The GPU
provides hardware acceleraon for 2D and 3D graphics by including one geometry processor
(GP) and two pixel processors (PP0 and PP1), each having a dedicated memory management unit
(MMU). The cache coherency between the APU and the GPU is achieved by cache-coherent
interconnect (CCI), which supports the AXI coherency extension (ACE) only.
CCI in-turn connects the APU and the GPU to the DDR controller, which arbitrates the DDR
access.
The following gure shows the mulmedia stack.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 38
Figure 7: Multimedia Stack
Graphics Application
Display Server
Ex: XII, Wayland
Mali common
Libraries
Video Codecs
Graphic Libraries
Ex: Open GLES1,
Open GLES 2, Open
VG
Gstreamer
Frame Buffer
Driver
Display Drivers
Video Drivers
Ex: V4L2
Mali Graphic
Drivers
DRM
APU
Linux Kernel Drivers
DDR Controller
Memory
GPO PPO PP1
Display Port
ARM MALI GPU
Cache Coherent
Interconnect
Ffmpeg pipeline
X14795-110320
The Linux kernel drivers for mulmedia enables the hardware access by the applicaons running
on the processors.
The following table lists the mulmedia drivers through the middleware stack that consists of the
libraries and framework components the applicaons use.
Table 11:
Libraries and Framework Components
Component Description
Display server Coordinates the input and output from the applications to
the operating system.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 39
Table 11: Libraries and Framework Components (cont'd)
Component Description
Graphics library The Zynq UltraScale+ MPSoC architecture supports OpenGL
ES 1.1 and 2.2, and Open VG 1.1.
Mali™-400 MP2 common libraries Mali-400 MP2 graphic libraries. For more details on how to
switch between different EGL backends, refer to Xilinx MALI
Driver.
Gstreamer A freeware multimedia framework that allows a
programmer to create a variety of media handling
components.
Video codecs Video encoders and decoders.
The following table lists the Linux kernel graphics drivers.
Table 12: Linux Kernel Drivers
Drivers Description
Frame buffer driver Kernel graphics driver exposing its interface through /dev/
fb*. This interface implements limited functionality
(allowing you to set a video mode and drawing to a linear
frame buffer).
Direct rendering manager (DRM) Serves in rendering the hardware between multiple user
space components.
Mali-400 MP2 graphics drivers Provides the hardware access to the GPU hardware.
Video drivers Video capture and output device pipeline drivers based on
the V4L2 framework. The Xilinx Linux V4L2 pipeline driver
represents the whole pipeline with multiple sub-devices.
You can configure the pipeline through the media node, and
you can perform control operations, such as stream on/off,
through the video node.
Device nodes are created be the pipeline driver. The
pipeline driver also includes the wrapper layer of the DMA
engine API, and this enables it to read/write frames from
RAM.
Display port drivers
Enables the hardware access to the display port, based on
DRM framework.
FreeRTOS Software Stack
Xilinx provides a FreeRTOS board support package (BSP) as a part of the Vis™ soware
plaorm. The FreeRTOS BSP provides you a simple, mul-threading environment with basic
features such as, standard input/output and access to processor hardware features. The BSP and
the included libraries are highly congurable to provide you the necessary funconality with the
least overhead. The FreeRTOS soware stack is similar to the bare metal soware stack, except
that it contains the FreeRTOS library. Xilinx device drivers included with the standalone libraries
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 40
can typically be used within FreeRTOS provided that only a single thread requires access to the
device. Xilinx bare metal drivers are not aware of Operang Systems. They do not provide any
support for mutexes to protect crical secons, nor do they provide any mechanism for
semaphores to be used for synchronizaon. While using the driver API with FreeRTOS kernel,
you must take care of this aspect.
The following gure illustrates the FreeRTOS soware stack for RPU.
Figure 8: FreeRTOS Software Stack
X16911-062121
User Applications
Lwip Networking File system (xilffs)
Secure Key
(xilkey)
Standard ‘C’
Library(libxil.a)
Display
Driver
ZDMA
drivers
Ethernet
Driver
USB
Driver
SD card
Driver
Flash
Drivers
SPI, I2C,
UART
Drivers
SYSMON
Drivers
Libraries
Drivers
Secure
(xilsecure)
Power Mgr API
(xilpm)
OpenAmp
(xilopenamp)
Parallel Flash
(xilflash)
ARM Cortex-R5 Core 0
RPU
FreeRTOS
ARM Cortex-R5 Core 1
Note: The FreeRTOS soware stack for APU is same as that for RPU except that the libraries support both
32-bit and 64-bit for APU.
Third-Party Software Stack
Many other embedded soware soluons are also available from the Xilinx partner ecosystem.
More informaon is available from the Xilinx website, Embedded Compung and the website,
Xilinx Third Party Tools.
Chapter 4: Software Stack
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 41
Chapter 5
Software Development Flow
This chapter explains the bare metal soware development for RPU and APU using the Vis™
unied soware plaorm as well as Linux soware development for APU using PetaLinux tools
and the Vis soware plaorm.
The following gure depicts the top-level soware architecture of the
Zynq
®
UltraScale+™ MPSoC.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 42
Figure 9: Software Development Architecture
Software Development
Tools
IDE
Boot Loader
Ex: U-Boot
Power Management
Firmware
Hardware
Handoff
Build Tools
Compiler Assembler
Linker
Zynq UltraScale+ MPSoC Hardware
Quad core
ARM-A53 APU
ARM Mali400
GPU
Dual core ARM
Cortex-R5F RPU
Enhanced DSP
and AXI
Platform
Management
Unit
System
Configuration
and Security Unit
DDR Memory
Controller
Debugger Profiler
Simulator Flash Writer
Debug Tools
· Configure PS
· Integrate IP
· Export Hardware to SDK
Vivado
Software
Hardware
OS Kernel
Applications
Middleware Stack
(Ex: Graphics, File system)
System Software
(Ex: Hypervisor, OpenAMP)
Security Management
Software
Software Stack
Programmable
Logic
Peripherals
OS Kernel
Drivers
X14793-051519
Bare Metal Application Development
This secon assists you in understanding the design ow of bare metal applicaon development
for APU and RPU using the Vis soware plaorm. The following gure shows the top-level
design ow in the Vis soware plaorm. You can create a C or C++ standalone applicaon
project by using the New Applicaon Project wizard.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 43
To create a project, follow these steps:
1. Click File → New → Applicaon Project. The New Applicaon Project dialog box appears.
Note: This is equivalent to clicking on File → New → Project to open the New Project wizard, selecng
Xilinx → Applicaon Project, and clicking Next.
2. Type a project name into the Project Name eld.
3. Select the locaon for the project. You can use the default locaon as displayed in the
Locaon eld by leaving the Use default locaon check box selected. Otherwise, click the
check box and type or browse to the directory locaon.
4. Select Create a new plaorm from hardware (XSA). The Vis IDE lists the all the available
pre-dened hardware designs.
5. Select any one hardware design from the list and click Next.
6. From the CPU drop-down list, select the processor for which you want to build the
applicaon. This is an important step when there are mulple processors in your design. In
this case you can either select psu_cortexa53_0 or psu_cortexr5_0.
7. Select your preferred language: C or C++.
8. Select an OS for the targeted applicaon.
9. Click Next to advance to the Templates screen.
The Vis soware plaorm provides useful sample applicaons listed in Templates dialog box
that you can use to create your project. The Descripon box displays a brief descripon of
the selected sample applicaon. When you use a sample applicaon for your project, the
Vis soware plaorm creates the required source and header les and linker script.
10. Select the desired template. If you want to create a blank project, select Empty Applicaon.
You can then add C les to the project, aer the project is created.
11. Click Finish to create your applicaon project and board support package (if it does not exist).
Note:
1. Xilinx recommends that you use the Managed Make ow rather than Standard Make C/C++ unless you
are comfortable working with make les.For more details on QEMU, see the Xilinx Quick Emulator User
Guide: QEMU.
2. Cortex
®
-R5F and Cortex
®
-A53 32-bit bare metal soware do not support 64-bit addressed data
transfer using device DMA.
3. By default, all standalone applicaons will run only on APU0. The other APU cores will be o.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 44
Application Development Using PetaLinux
Tools
Soware development ow in the PetaLinux tools environment involves many stages. To simplify
understanding, the following gure shows a chart with all the stages in the PetaLinux tools
applicaon development.
Figure 10: PetaLinux Tool-Based Software Development Flow
Petalinux Tools
Build Tools
Debug and Profile Tools
GNU Petalinux-Build
Yocto Make
GDB Petalinux-Boot
QEMU OProfile
X14815-063017
Linux Application Development Using Vitis
Xilinx soware design tools facilitate the development of Linux user applicaons. This secon
provides an overview of the development ow for Linux applicaon development.
The following gure illustrates the typical steps involved to develop Linux user applicaons using
the Vis soware plaorm.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 45
Figure 11: Linux Application Development Flow
Invoke SDK
Create a Linux application
project
Build the Linux application
project
Software Application
Development
Profiling
Performance met?
Download Hardware
Bitstream to FPGA
Debug
Functionality
achieved?
Open / Create
SDK workspace
Adding an Application to
Linux file system
Yes
Yes
No
No
Boot Linux & set up target
connection
X14816-063017
Creating a Linux Application Project
You can create a C or C++ Linux applicaon project by using the New Applicaon Project wizard.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 46
To create a project:
1. Click File → New → Applicaon Project. The New Applicaon Project dialog box appears.
2. Type a project name into the Project Name eld.
3. Select the locaon for the project. You can use the default locaon as displayed in the
Locaon eld by leaving the Use default locaon check box selected. Otherwise, click the
check box and type or browse to the directory locaon.
4. Select Next.
5. On the Select plaorm tab, select the Plaorm that has a Linux domain and click Next.
6. On the Domain window, select the domain from the Domain drop-down.
7. Select your preferred language: C or C++.
8. Oponally, select Linux System Root to specify the Linux sysroot path and select Linux
Toolchain to specify the Linux toolchain path.
9. Click Next to move to the Templates screen.
10. The Vis soware plaorm provides useful sample applicaons listed in the Templates dialog
box that you can use to create your project. The Descripon box displays a brief descripon
of the selected sample applicaon. When you use a sample applicaon for your project, the
Vis soware plaorm creates the required source and header les and linker script.
11. Select the desired template. If you want to create a blank project, select Empty Applicaon.
You can then add C les to the project, aer the project is created.
12. Click Finish to create your Linux applicaon project.
13. Click the
icon to generate or build the applicaon project.
Create a Hello World Application
Aer installing the Vis™ soware plaorm, the next step is to create a soware applicaon
project. Soware applicaon projects are the nal applicaon containers. The project directory
that is created contains (or links to) your C/C++ source les, executable output le, and
associated ulity les, such as the Makeles used to build the project.
Note: The Vis soware plaorm automacally creates a system project for you. A system project is a top-
level container project that holds all of the applicaons that can run in a system at the same me. This is
useful if you have many processors in your system, especially if they communicate with one another,
because you can debug, launch, and prole applicaons as a set instead of as individual items.
Build a Sample Application
This secon describes how to create a sample Hello World applicaon using an exisng template.
1. Launch the Vis soware plaorm.
2. Select a workspace directory for your rst project.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 47
3. Click Launch. The welcome page appears.
4. Close the welcome page. The development perspecve opens.
5. Select File → New → Applicaon Project.
6. Enter a name in the Project name eld and click Next. The Select plaorm tab opens. You
should choose a plaorm for your project. You can either use a pre-supplied plaorm (from
Xilinx or another vendor), a previously created custom plaorm, or you can create one
automacally from an exported Vivado
®
hardware project.
7. On the Select plaorm tab, click the plaorm you just created and click Next. To use your
own hardware plaorm, click the
icon and add your plaorm to the list.
8. Select the system conguraon for your project and click Next. The Templates window
opens.
9. Select Hello World and click Next. Your workspace opens with the Explorer pane showing
the hello_world_system system project and the zcu102 plaorm project.
10. Right-click the system project and select Build Project. You have now built your applicaon
and the Console tab shows the details of the le and applicaon size.
Debug and Run the Application
Now that you have generated the executable binary, you can test it on a board. To run the
applicaon on the board, perform the following preliminary steps:
• Connect a JTAG cable to the computer.
• Set the Boot Mode switch of the board to JTAG mode.
• Connect a USB UART cable and setup your UART console.
• Power up the board.
1. Expand the system project and choose the applicaon project you want to debug. Right-click
the applicaon and select Debug As → Launch on Hardware (Single Applicaon Debug).
2. On the Conrm Perspecve Switch dialog, click Yes. The Vis IDE switches to the Debug
perspecve and the debugger stops at the entry to your main() funcon.
3. Using the commands in the toolbar, step through the applicaon. Aer you step through the
print() funcon, Hello World appears in the UART console.
Adding Driver Support for Custom IP in the PL
The Vis soware plaorm supports Linux BSP generaon for peripherals in the PS as well as
custom IP in the PL. When generang a Linux BSP, the Vis soware plaorm produces a device
tree, which is a data structure describing the hardware system that passes to the kernel when
you boot.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 48
Device drivers are available as part of the kernel or as separate modules, and the device tree
denes the set of hardware funcons available and features enabled.
Addionally, you can add dynamic, loadable drivers. The Linux kernel supports these drivers.
Custom IP in the PL are highly congurable, and the device tree parameters dene both the set
of IP available in the system and the hardware features enabled in each IP.
See Chapter 3: Development Tools for addional overview informaon on the Linux Kernel and
boot sequence.
Chapter 5: Software Development Flow
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 49
Chapter 6
Software Design Paradigms
The Xilinx
®
Zynq
®
UltraScale+™ MPSoC architecture supports heterogeneous mulprocessor
engines targeted at dierent tasks. The main approaches for developing soware to target these
processors are by using the following:
• Frameworks for Mulprocessor Development: Describes the frameworks available for
development on the Zynq UltraScale+ MPSoC.
• Symmetric Mulprocessing (SMP): Using SMP with PetaLinux is the most simple ow for
developing an SMP with a Linux plaorm for the Zynq UltraScale+ MPSoC.
• Asymmetric Mulprocessing (AMP): AMP is a powerful mode to use mulple processor
engines with precise control over what runs on each processor. Unlike SMP, there are many
dierent ways to use AMP. This secon describes two ways of using AMP with varying levels
of complexity.
The following secons describe these development methods in more detail.
Frameworks for Multiprocessor Development
Xilinx provides mulple frameworks for Zynq UltraScale+ MPSoCs to facilitate the applicaon
development on the heterogeneous processors and Xilinx 7 series FPGAs. These frameworks are
described as follows:
• Hypervisor Framework: Xilinx provides the Xen hypervisor, a crical item needed to support
virtualizaon on APU of Zynq UltraScale+ MPSoC. The Use of Hypervisors secon covers
more details.
• Authencaon Framework: The Zynq UltraScale+ MPSoC supports authencaon and
encrypon features as a part of authencaon framework. To understand more about the
authencaon framework, see Boot Time Security.
• TrustZone Framework: The TrustZone technology allows and maintains isolaon between
secure and non-secure processes within the same system. See this whitepaper for more
informaon.
Xilinx provides the trustzone support through the Trusted Firmware-A (TF-A) to maintain the
isolaon between secure and non-secure worlds. To understand more about TF-A, see Trusted
Firmware-A.
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 50
• Mulprocessor Communicaon Framework: Xilinx provides the OpenAMP framework for
Zynq UltraScale+ MPSoCs to allow communicaon between the dierent processing units.
For more details, see the Xilinx Quick Emulator User Guide: QEMU.
• Power Management Framework: The power management framework allows soware
components running across dierent processing units to communicate with the power
management unit.
Symmetric Multiprocessing (SMP)
SMP enables the use of mulple processors through a single operang system instance. The
operang system handles most of the complexity of managing mulple processors, caches,
peripheral interrupts, and load balancing.
The APU in the Zynq UltraScale+ MPSoCs contains four homogeneous cache coherent Arm
®
Cortex
®
-A53 processors that support SMP mode of operaon using an OS (Linux or VxWorks).
Xilinx and its partners provide operang systems that make it easy to leverage SMP in the APU.
The following diagram shows an example of Linux SMP with mulple applicaons running on a
single OS.
Figure 12: Example SMP Using Linux
APU
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Linux Kernel in SMP
Application n
Thread 1
Thread 2
Thread n
Application 1
Thread 1
Thread 2
Thread n
X14837-063017
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 51
This would not be the best mode of operaon when there are hard, real-me requirements as it
ignores Linux applicaon core anity which should be available to developers with the exisng
Xilinx soware.
Asymmetric Multiprocessing (AMP)
AMP uses mulple processors with precise control over what runs on each processor. Unlike
SMP, there are many dierent ways to use AMP. This secon describes two ways of using AMP
with varying levels of complexity.
In AMP, a soware developer must decide what code has to run on each processor before
compiling and creang a boot image that includes the soware executable for each CPU. Using
AMP with the Arm Cortex-R5F processors (SMP is not supported in Cortex-R5F) in the RPU
enables developers to meet highly demanding, hard real-me requirements as opposed to so
real-me requirements.
You can develop the applicaons independently, and program those applicaons to communicate
with each other using inter-processing communicaon (IPC) opons. See this link to the
“Interrupts” chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085) for
further descripon of this feature.
You can also apply this AMP method to applicaons running on MicroBlaze™ processors in the
PL or even in the APU. The following diagram shows an AMP example with applicaons running
on the RPU and the PL without any communicaon with each other.
Figure 13: AMP Example using Bare-Metal Applications Running on RPU and PL
MicroBlaze
Bare-metal
Application
Bare-metal
Application
Bare-metal
Application
RPU
PL
Arm
Cortex-R5F
Arm
Cortex-R5F
MicroBlaze
MicroBlaze
X19225-071317
OpenAMP
The OpenAMP framework provides mechanisms to do the following:
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 52
• Load and unload rmware
• Communicate between applicaons using a standard API
The following diagram shows an example of an OpenAMP and the hard real-me capabilies of
the RPU using the OpenAMP framework.
In this case, Linux applicaons running on the APU perform the loading and unloading of RPU
applicaons. This allows developers to load dierent processing dedicated algorithms to the RPU
processing engines as needed with very determinisc performance.
Figure 14: Example with SMP and AMP using OpenAMP Framework
X14839-063017
APU
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Linux Kernel in SMP mode
RTOS
Kernel
Baremetal
Application
Application 1
Thread 1
Thread 2
Thread n
RPU
Arm
Cortex-R5F
Arm
Cortex-R5F
PL
MicroBlaze
MicroBlaze
RTOS
Kernel
Baremetal
Application
Baremetal
Application
Application n
Thread 1
Thread 2
Thread n
RPMsg
Open AMP
(APIs for loading/ Unloading
firmware, Message Passing)
RPMsg
See the Libmetal and OpenAMP for Zynq Devices User Guide (UG1186) for more informaon about
the OpenAMP Framework.
Virtualization with Hypervisor
The Zynq UltraScale+ MPSoCs include a hardware virtualizaon extension on the Arm Cortex-
A53 processors, interrupt controller, and Arm System MMU (SMMU) that provides exibility to
combine various operang system combinaons, including SMP and AMP, within the APU.
The following diagram shows an example of an SMP-capable OS, like Linux working along with
Real-Time Operang System (RTOS) as well as a bare metal applicaon using a single hypervisor.
This enables independent development of applicaons in their respecve mode of operaon.
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 53
Figure 15: Example with Hypervisor
X14840-063017
Hypervisor
APU
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Arm
Cortex-A53
Linux Kernel in
SMP mode
RTOS
Kernel
Baremetal
Application
Application 1
Thread 1
Thread 2
Thread n
Application n
Task 1
Task 2
Task n
Although the hardware virtualizaon included within Zynq UltraScale+ MPSoC and its
hypervisors allow the standard operang systems and their applicaons to funcon with low to
moderate eort, the addion of a hypervisor does bring design complexity to low-level system
funcons such as power management, FPGA bitstream management,
OpenAMP soware stack, and security accelerator access which must use addional underlying
layers of system rmware. Hence, Xilinx recommends that the developers must iniate early
eort into these aspects of system architecture and implementaon.
For more details on using Hypervisor like the Xen Hypervisor, see the MPSoC Xen Hypervisor
website.
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 54
Use of Hypervisors
Xilinx distributes a port for the Xen open source hypervisor in the Xilinx
Zynq UltraScale+ MPSoC. Xen hypervisor provides the ability to run mulple operang systems
on the same compung plaorm. Xen hypervisor, which runs directly on the hardware, is
responsible for managing CPU, memory, and interrupts. Mulple numbers of OS can run on top
of the hypervisor. These operang systems are called domains (also called as virtual machines
(VMs)).
The Xen hypervisor provides the ability to concurrently run mulple operang systems and their
standard applicaons with relave ease. However, Xen does not provide a generic interface
which gives the guest an operang system access to system funcons. Hence, you need to follow
the cauons menoned in this secon.
Xen hypervisor controls one domain, which is domain 0, and one or more guest domains. The
control domain has special privileges, such as the following:
• Capability to access the hardware directly
• Ability to handle access to the I/O funcons of the system
• Interacon with other virtual machines.
It also exposes a control interface to the outside world, through which the system is controlled.
Each guest domain runs its own OS and applicaon. Guest domains are completely isolated from
the hardware.
Running mulple Operang Systems using Xen hypervisor involves seng up the host OS and
adding one or more guest OS.
Note: Xen hypervisor is available as a selectable component within the PetaLinux tools; Xen hypervisor can
also be downloaded from Xilinx GIT. With Linux and Xen soware that is made available by Xilinx, it is
possible to build custom Linux guest conguraons. Guest OS other than Linux require addional soware
and eort from third-pares. See the PetaLinux Product Page for more informaon.
Chapter 6: Software Design Paradigms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 55
Chapter 7
System Boot and Configuration
Zynq
®
UltraScale+™ MPSoCs support the ability to boot from dierent devices such as a QSPI
ash, an SD card, a host with Device Firmware Upgrade ulity installed on it, or a NAND ash in
place. This chapter details the boong process using dierent boong devices in both secure and
non-secure modes.
Boot Process Overview
The plaorm management unit (PMU) and conguraon security unit (CSU) manage and perform
the mul-staged boong. You can boot the device in either secure (using authencated boot
image) or non-secure (using an unauthencated boot image) mode. The boot stages are as
follows:
• Pre-conguraon stage: The PMU primarily controls pre-conguraon stage that executes
PMU ROM to setup the system. The PMU handles all of the processes related to reset and
wake-up.
• Conguraon stage: This stage is responsible for loading the rst-stage boot loader (FSBL)
code for the PS into the on-chip RAM (OCM). It supports both secure and non-secure boot
modes. Through the boot header, you can execute FSBL on the Cortex
®
-R5F-0 / R5-1
processor or the Cortex
®
-A53 processor. The Cortex-R5F-0 processor also supports lock step
mode.
• Post-conguraon stage: Aer FSBL execuon starts, the Zynq UltraScale+ MPSoC enters
the post conguraon stage.
Boot Flow
There are two boot ows in the Zynq UltraScale+ MPSoC architecture: secure and non-secure.
The following secons describe some of the example boot sequences in which you bring up
various processors and execute the required boot tasks.
Note: The gures in these secons show the complete boot ow, including all mandatory and oponal
components.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 56
Figure 16: Boot Flow Example
Time
Release
CSU
Power
Monitoring
Load
FSBL
Tamper
Monitoring
PMU
CSU
RPU
FSBL
APU
U-Boot
Linux
ATF
X18969-062421
Non-Secure Boot Flow
In non-secure boot mode, the PMU releases the reset of the conguraon security unit (CSU),
and enters the PMU server mode where it monitors power. Aer the PMU releases the CSU
from reset, it loads the FSBL into OCM. PMU rmware runs from PMU RAM in parallel to FSBL
in OCM. FSBL is run on APU or RPU. FSBL runs from APU/RPU and TF-A; U-Boot and Linux run
on APU. Other boot conguraons allow the RPU to start and operate wholly independent of
the APU and vice-versa.
• On APU, TF-A will be executed aer the FSBL hands o to TF-A. TF-A hands o to a second
stage boot loader like U-Boot which executes and loads an operang system such as Linux.
• On RPU, FSBL hands o to a soware applicaon.
• Linux, in turn, loads the executable soware.
Note: The operang system manages the mulple Cortex-A53 processors in symmetric mul-processing
(SMP) mode.
Secure Boot Flow
In the secure boot mode, the PMU releases the reset of the conguraon security unit (CSU) and
enters the PMU server mode where it monitors power. Aer the PMU releases the CSU from
reset, the CSU checks to determine if authencaon is required by the FSBL or the user
applicaon.
The CSU does the following:
• Performs an authencaon check and proceeds only if the authencaon check passes. Then
checks the image for any encrypted parons.
• If the CSU detects parons that are encrypted, the CSU performs decrypon and loads the
FSBL into the OCM.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 57
For more informaon on CSU, see the Conguraon Security Unit secon.
FSBL running on APU hands o to TF-A. FSBL running on RPU loads TF-A. In both the cases, TF-
A loads U-Boot which loads the OS. TF-A then executes the U-Boot and loads an OS such as
Linux. Then Linux, in turn, loads the executable soware. Similarly, FSBL checks for
authencaon and encrypon of each paron it tries to load. The parons are only loaded by
FSBL on successful authencaon and decrypon (if previously encrypted).
Note: In the secure boot mode, psu_coresight_0 is not supported as a stdout port.
Boot Image Creation
Bootgen is a tool that lets you stch binary les together and generate device boot images.
Bootgen denes mulple properes, aributes and parameters that are input while creang boot
images for use in a device.
The secure boot feature for devices uses public and private key cryptographic algorithms.
Bootgen provides assignment of specic desnaon memory addresses and alignment
requirements for each paron. It also supports encrypon and authencaon, described in the
Bootgen User Guide (UG1283). More advanced authencaon ows and key management opons
are discussed in the Using HSM Mode secon of Bootgen User Guide (UG1283), where Bootgen
can output intermediate hash les that can be signed oine using private keys to sign the
authencaon cercates included in the boot image. The program assembles a boot image by
adding header blocks to a list of parons.
Oponally, each paron can be encrypted and authencated with Bootgen. The output is a
single le that can be directly programmed into the boot ash memory of the system.
Various input les can be generated by the tool to support authencaon and encrypon as well.
Bootgen comes with both a GUI interface and a command line opon. The tool is integrated into
the soware development toolkit, Integrated Development Environment (IDE), for generang
basic boot images using a GUI, but the majority of Bootgen opons are command line-driven.
Command line opons can be scripted. The Bootgen tool is driven by a boot image format (BIF)
conguraon le, with a le extension of *.bif. Along with SoC, ACAP, and Bootgen has the
ability to encrypt and authencate parons for and later FPGAs, as described in FPGA Support.
In addion to the supported command and aributes that dene the behavior of a Boot Image,
there are ulies that help you work with Bootgen. Bootgen code is now available on Github.
Creating a Bootable Image
When a system project is selected, by running build, the Vis soware plaorm builds all
applicaons in the system project and creates a bootable image according to a pre-dened BIF or
an auto-generated BIF.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 58
You can create bootable images using Bootgen. In the Vis™ IDE, the Create Boot Image menu
opon is used to create the boot image.
To create a bootable image, follow these steps:
1. Select the Applicaon Project in the Project Explorer view.
2. Right-click the applicaon and select Create Boot Image to open the Create Boot Image
dialog box.
3. Specify the boot loader and the parons.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 59
4. Click Create Image to create the image and generate the BOOT.bin in the
<Application_project_name>/_ide/bootimage folder.
Boot Modes
Refer to this link to the “Boot and Conguraon” chapter of the Zynq UltraScale+ Device Technical
Reference Manual (UG1085) for a comprehensive table of the available boot modes.
QSPI24 and QSPI32 Boot Modes
The QSPI24 and QSPI32 boot modes support the following:
• x1, x2, and x4 read modes for single Quad SPI ash memory 24 (QSPI24) and single Quad SPI
ash memory 32 (QSPI32)
• x8 read mode for dual QSPI.
• Image search for MulBoot
• I/O mode for BSP drivers (no support in FSBL)
The bootROM searches the rst 256 Mb in x8 mode. In QSPI24 and QSPI32 boot modes (where
the QSPI24/32 device is < 128 Mb), to use MulBoot, place the mulple images so that they t
in memory locaons less than 128 Mb. The pin conguraon for QSPI24 boot mode is 0x1.
Note: QSPI dual stacked (x8) boot is not supported. Only QSPI Single Transmission Rate (STR) is supported.
Single Quad-SPI memory (x1, x2, and x4) is the only boot mode that supports execute-in-place (XIP).
To create a QSPI24/QSPI32 boot image, provide the following les to the Bootgen tool:
• An FSBL ELF
• A secondary boot loader (SBL), such as U-Boot, or a Cortex-R5F-0/R5-1 and/or a Cortex-A53
applicaon ELF
• Authencaon and encrypon key (oponal)
For more informaon on Authencaon and Encrypon, see Chapter 8: Security Features.
Bootgen generates the boot.mcs and a boot.bin binary le that you can write into the QSPI
ash using the ash writer. MCS is an Intel hex-formaed le that includes a checksum for
reliability.
Note: The pin conguraon for QSPI24 boot mode is 0x1 for qspi 24 and 0x2 for qspi32.
SD Boot Mode
SD boot (version 3.0) supports the following:
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 60
• FAT 16/32 le systems for reading the boot images.
• Image search for MulBoot with a maximum number 8,192 les are supported.
Note: exFAT is not supported for SD boot in FSBL and U-Boot.
The following gure shows an example for boong Linux in SD mode.
Figure 17: Booting in SD Mode
Bootgen Tool
Boot.bin
FAT 32
Kernel Image
Device tree file
Root file system
EXT 3
File system
Zynq
UltraScale+
MPSoc
0 1 0 1
Petalinux SDK
SDK
FSBL
U-Boot
A53 Image
KEY
.bif
SD card
Boot Mode
pins
KEY
Board
X14933-063017
To create an SD boot image, provide the following les to Bootgen:
• An FSBL ELF
• A Cortex-R5F-0/R5-1 and/or an Cortex-A53 applicaon ELF
• Oponal authencaon and encrypon keys
The Bootgen tool generates the boot.bin binary le. You can write the boot.bin le into an SD
card using a SD card reader.
In PetaLinux, do the following:
1. Build the Linux kernel image, device tree le, and the root le system.
2. Copy the les into the SD card.
The formaed SD card then contains the boot.bin, the kernel image, and the device tree le in
the FAT32 paron; the root le system resides in the EXT 4 paron.
IMPORTANT!
To boot from SD1, congure the boot pins to
0x5
. To boot from SD0, congure the boot
pins to
0x3
.To boot from SD with a level shier, congure the boot pins to
0xE
.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 61
eMMC18 Boot Mode
eMMC18 boot (version 4.5) supports the following:
• FAT 16/32 le systems for reading the boot images.
• Image search for MulBoot with a maximum number of 8,192 les being supported.
The following gure shows an example for boong Linux in eMMC18 mode.
Figure 18: Booting in eMMC18 Mode
Bootgen Tool
Boot.bin
FAT 32
Kernel Image
Device tree file
Root file system
EXT 3
File system
Zynq
UltraScale+
MPSoc
0 1 1 0
PetaLinux SDK
SDK
FSBL
U-Boot
A53 Image
KEY
.bif
eMMC18 card
Boot Mode
pins
KEY
Board
X18971-071317
To create an eMMC18 boot image, provide the following les to Bootgen:
• An FSBL ELF
• A Cortex
®
-R5F-0/R5-1 and/or a Cortex
®
-A53 applicaon ELF
• Oponal authencaon and encrypon keys
The Bootgen tool generates the boot.bin binary le. You can write the boot.bin le into an
eMMC18 card using an eMMC18 card reader.
In PetaLinux, do the following:
• Build the Linux kernel image, device tree le, and the root le system.
• Copy the les into the eMMC18 card.
The formaed eMMC18 card then contains the boot.bin, the kernel image, and the device tree
les in the FAT32 paron; the root le system resides in the EXT4 paron.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 62
NAND Boot Mode
The NAND boot only supports 8-bit widths for reading the boot images, and image search for
MulBoot. The following gure shows an example for boong Linux in NAND mode.
Figure 19: Booting in NAND Mode
Boot Header
FSBL
SBL
R5 Image
A53 Image
Bootgen
Tool
Flash
Writer
Flash
Boot.bin
eFuse
FSBL
R5 Image
A53 Image
.bif
SDK
JTAG
Key
QSPI
Board
NAND
Converter
NAND
image
Key
FSBL
Boot Pins
Zynq
UltraScale+
MPSoC
0100
X14934-071317
To create a NAND boot image, provide the following les to Bootgen:
• An FSBL ELF
• A Cortex-R5F-0/R5-1 applicaon ELF and/or an Cortex-A53 applicaon ELF
• Oponal authencaon/encrypon keys
The Bootgen tool generates the boot.bin binary le. You can then write the NAND bootable
image into the NAND ash using the ash writer
IMPORTANT!
To boot from NAND, congure boot pins to
0x4
.
JTAG Boot Mode
You can manually download any soware image needed for the PS and any hardware image on
the PL using JTAG. For JTAG boot mode sengs, see this link in the Zynq UltraScale+ Device
Technical Reference Manual (UG1085).
IMPORTANT!
Secure boot is not supported in the JTAG mode.
USB Boot Mode
The USB boot mode supports only USB 2.0. In USB boot mode, both the secure and non-secure
boot modes are supported. USB boot mode is not supported for DDR-less systems. Features like
Mulboot, fallback image, and XIP are not supported.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 63
Note: USB boot mode is disabled by default in FSBL. To enable the USB boot mode, congure the
FSBL_USB_EXCLUDE_VAL to 0 in xfsbl_config.h le.
Table 13: USB Boot Mode Details
Pin Functionality
Mode pins 0x7
MIO pins MIO[63:52]
Non-secure Yes
Secure Yes
Signed Yes
Mode Slave
USB boot mode requires a host PC with dfu-uls installed on it. The host and device need to be
connected through a USB 2.0 or USB 3.0 cable. The host must contain one boot.bin to be loaded
by bootROM, which contains only fsbl.elf and another boot_all.bin to be loaded by
FSBL. On powering up the board in USB boot mode, issue the following commands:
• On Linux host:
• dfu-ul -D boot.bin: This downloads the le to the device, which is processed by
bootROM.
• dfu-ul -D boot_all.bin: This downloads the le to the device, which is processed by FSBL.
• On Windows host:
• dfu-ul.exe -D boot.bin: This downloads the le to the device, which is processed by
bootROM.
• dfu-ul.exe -D boot_all.bin: This downloads the le to the device, which is then processed
by FSBL.
The size limit of boot.bin and boot_all.bin are the sizes of OCM and DDR. The size of
OCM is 256 KB.
Secondary Boot Mode
There is a provision to have two boot devices in the Zynq UltraScale+ MPSoC architecture. The
primary boot mode is the boot mode used by bootROM to load FSBL and oponally PMU FW.
The secondary boot mode is the boot device used by FSBL to load all the other parons. The
supported secondary boot modes are QSPI24, QSPI32, SD0, eMMC, SD1, SD1-LS, NAND and
USB.
When using PS-PCIe
®
on ZU+ in Endpoint mode, running FSBL is enough to set up the block for
endpoint mode operaon. FSBL should be able to program the PS/PS-PCIe
®
and GTR within 100
ms. However, this doesn’t include PL-bitstream programming as including that would make this
greater than 100 ms.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 64
IMPORTANT! If secondary boot mode is specied, it should be dierent from the primary boot device. For
example, if QSPI32 is the primary boot mode, QSPI24 cannot be the secondary boot mode. Instead, you
can have SD0, eMMC, SD1, SD1-LS, NAND, USB as secondary boot modes. All combinaons of boot
devices are supported as primary and secondary boot devices.
Note: By default, the secondary boot mode is the same as primary boot mode and there will be only one
boot image.
See What is Secondary Boot Mode in FSBL wiki page for more informaon.
Detailed Boot Flow
The plaorm management unit (PMU) in the Zynq UltraScale+ MPSoC is responsible for handling
the primary pre-boot tasks.
PMU ROM will execute from a ROM during boot to congure a default power state for the
device, inialize RAMs, and test memories and registers. Aer the PMU performs these tasks and
relinquishes system control to the conguraon security unit (CSU), it enters a service mode. In
this mode, the PMU responds to interrupt requests made by system soware through the
register interface or by hardware through the dedicated I/O to perform plaorm management
services.
Pre-Boot Sequence
The following table lists the tasks performed by the PMU in the pre-Boot sequence.
Table 14: Pre-Boot Sequence
Pre-Boot Task Description
0 Initialize MicroBlaze™ processor. Capture key states.
1 Scan, and clear LPD and FPD.
2 Initialize the System Monitor.
3 Initialize the PLL used for MBIST clocks.
4 Zero out the PMU RAM.
5 Validate the PLL. Configure the MBIST clock.
6 Validate the power supply.
7 Repair FPD memory (if required).
8 Zeroize the LPD and FPD and initialize memory self-test.
9 Power-down any disabled IPs.
10 Either release CSU or enter error state.
11 Enter service mode.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 65
As soon as the CSU reset is released, it executes the CSU bootROM and performs the following
sequence:
1. Inializes the OCM.
2. Determines the boot mode by reading the boot mode register, which captures the boot-mode
pin strapping at the POR.
3. The CSU connues with the FSBL load and the oponal PMU rmware load. PMU rmware
is the soware that can be executed by the PMU unit. The code executes from the RAM of
the PMU. See Chapter 9: Plaorm Management for more informaon.
The following gure shows the detailed boot ow diagram.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 66
Figure 20: Detailed Boot Flow Example
Power ON
PMU releases CSU Reset
Read Boot Mode Pins
Read Boot Header from
the Boot image
Check for
Authentication
Fallback bootDecrypt FSBL
Load FSBL to OCM
FSBL configures the PS
FSBL configures PL with
the bitstream
FSBL loads the RPU
software
FSBL loads the APU
software
FSBL Handoff to APU
software
Is FSBL
Authenticated?
YES
YES
NO
YES
NO
Is FSBL
Encrypted?
Authentication
Test passed?
Decryption
Fails?
NO
Fallback boot
YES
APU/RPU
CSU
PMU
X14935-070717
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 67
Disabling FPD in Boot Sequence
Perform the following to avoid an FPD lockout, where FPD Power is applied momentarily:
• Apply the power unl the compleon of bootROM execuon.
• To power down the FP during FSBL execuon, set FPD bit '22' of PMU_GLOBAL
REQ_PWRDWN_STATUS register.
• To bring the FP domain up in a later stage of the boot process, set the PMU_GLOBAL
REQ_PWRUP_STATUS bit to '22’.
Perform the following in cases where the FPD power is not applied before the FSBL boots
1. Power up the R5.
2. A register is set indicang the FPD is locked pending POR as the reset or clear sequence
cannot execute on the FPD.
3. R5 can read the FP locked status from PMU_GLOBAL REQ_ISO_STATUS register bit ‘4’.
4. At this stage, PMU_GLOBAL REQ_PWRUP_STATUS bit '22' will not be set.
5. To bring the FPD node back up, power must be supplied to the node and a POR needs to be
issued.
Setting FSBL Compilation Flags
You can set compilaon ags using the C/C++ sengs in the Vis FSBL project, as shown in the
following gure:
Note: There is no need to change any of the FSBL source les or header les to include these ags.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 68
Figure 21: FSBL Debug Flags
The following table lists the FSBL compilaon ags.
Table 15: FSBL Compilation Flags
Flag Description
FSBL_DEBUG Prints basic information and error prints, if any.
FSBL_DEBUG_INFO Enables debug information in addition to the basic information.
FSBL_DEBUG_DETAILED Prints information with all data exchanged.
FSBL_NAND_EXCLUDE Excludes NAND support code.
FSBL_QSPI_EXCLUDE Excludes QSPI support code.
FSBL_SD_EXCLUDE Excludes SD support code.
SBL_SECURE_EXCLUDE Excludes authentication code and encryption code but not checksum
code.
FSBL_BS_EXCLUDE Excludes bitstream code.
FSBL_EARLY_HANDOFF_EXCLUDE Excludes early handoff related code.
FSBL_WDT_EXCLUDE Excludes WDT support code.
FSBL_PERF_EXCLUDE_VAL Excludes performance prints.
FSBL_A53_TCM_ECC_EXCLUDE_VAL Excludes TCM ECC Init for A53.
FSBL_PL_CLEAR_EXCLUDE_VAL Excludes PL clear unless boot.bin contains bitstream.
FSBL_PROT_BYPASS_EXCLUDE_VAL Excludes isolation configurations.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 69
Table 15: FSBL Compilation Flags (cont'd)
Flag Description
8FSBL_PARTITION_LOAD_EXCLUDE_VAL Excluded partition loading.
FSBL_USB_EXCLUDE_VAL Excludes USB code.
This is set to 1 by default.
Set this value to 0 to enable USB boot mode.
FSBL_FORCE_ENC_EXCLUDE_VAL Excludes forcing encryption of all partitions when ENC_ONLY fuse is
programmed. By default, this is set to 0.
FSBL forces to enable encryption for all the partitions when ENC_ONLY
is programmed.
FSBL_DDR_SR_EXCLUDE_VAL DDR Excludes self refresh code.
FSBL_TPM_EXCLUDE_VAL Excludes TPM related code.
FSBL_PL_LOAD_FROM_OCM_EXCLUDE_VAL Excludes the code to load bitstream in chunks from OCM.
FSBL_UNPROVISIONED_AUTH_SIGN_EXCLUDE_VA
L
Excludes the code to load authenticated partitions as non-secure when
EFUSEs are not programmed and when boot header is not
authenticated.
See "I’m unable to build FSBL due to size issues, how can I reduce its footprint?" secon in FSBL
wiki page for more informaon.
Enabling Debug Prints
See FSBL wiki page for more informaon on debugging FSBL.
Fallback and MultiBoot Flow
In the Zynq
®
UltraScale+™ MPSoC, the CSU bootROM supports MulBoot and fallback boot
image search where the conguraon security unit CSU ROM or bootROM searches through the
boot device looking for a valid image to load. The sequence is as follows:
• BootROM searches for a valid image idencaon string (XLNX as image ID) at osets of 32
KB in the ash.
• Aer nding a valid idencaon value, validates the checksum for the header.
• If the checksum is valid, the bootROM loads the image. This allows for more than one image in
the ash.
In MulBoot:
• CSU ROM or FSBL or the user applicaon must iniate the boot image search to choose a
dierent image from which to boot.
• To iniate this image search, CSU ROM or FSBL updates the MulBoot oset to point to the
intended boot image, and generates a so reset by wring into the CRL_APB register.
The following gure shows an example of the fallback using the MulBoot ow.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 70
Figure 22: Fallback using the MultiBoot Flow
Boot Header
Image 1
.
.
.
.
Boot Header
Image 2
0x100_0000
0x002_0000
0x000_0000
Boot Header 1
Image 1
.
.
.
.
.
Boot Header 2
Image 2
0x100_0000
0x002_0000
0x000_0000
.
.
.
.
0x001_0000
0x000_8000
0x001_8000
Multi-Boot
Offset=1
Multi-Boot
Offset=2
Multi-Boot
Offset=3
Multi-Boot
Offset=4
X14936-071217
Note: The same ow is applicable to both Secure and Non-secure boot methods.
In the example fallback boot ow gure, the following sequence occurs:
• Inially, the CSU bootROM loads the boot image found at 0x000_0000.
• If this image is found to be corrupted or the decrypon and authencaon fails, CSU
bootROM increments the MulBoot oset by one and searches for a valid boot image at
0x000_8000 (32 KB oset).
• If the CSU bootROM does not nd the valid idencaon value, it again increments the
MulBoot oset by 1, and searches for a valid boot image at the next 32 KB aligned address.
• The CSU bootROM repeats this unl a valid boot image is found or the image search limit is
reached. In this example ow, the next image is shown at 0x002_0000 corresponding to a
MulBoot oset value of four.
• In the example MulBoot ow, to load the second image that is at the address 0x002_0000,
MuBoot oset is updated to four by FSBL/CSU-ROM. When the MulBoot oset is
updated, so reset the system.
The following table shows the MulBoot image search range for dierent boong devices.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 71
Table 16: Boot Devices and MultiBoot Image Search Range
Boot Device MultiBoot Image Search Range
QSPI Single (24-bit) 16 MB
QSPI Dual (24-bit) 32 MB
QSPI Single (32-bit) 256 MB
QSPI Dual (32-bit) 512 MB
NAND 128 MB
SD/EMMC 8,191 boot files
USB Not applicable
FSBL Build Process
Aer authencang and/or decrypng, the FSBL is loaded into OCM and handed o by the CSU
bootROM. First Stage Boot Loader congures the FPGA with a bitstream (if it exists) and loads
the Standalone (SA) Image or Second Stage Boot Loader image from the non-volale memory
(NAND/SD/eMMC/QSPI) to RAM(DDR/TCM/OCM). It takes the Cortex-R5F-0/R5F-1 processor
or the Cortex-A53 processor unit out of reset. It supports mulple parons. Each paron can
be a code image or a bitstream. Each of these parons, if required, will be authencated and/or
decrypted.
Note: If you are creang a custom FSBL, you should be aware that the OCM size is 256 KB and is available
to CSU bootROM. The FSBL size is close to 170 KB and it would t in the OCM. While using the USB boot
mode, you should make sure that the PMU rmware is loaded by the FSBL and not by the CSU bootROM.
This is because the size of boot.bin loaded by the CSU bootROM should be less than 256 KB.
Note: Users can load bitstream from OCM for non-encrypted cases even if DDR is present in the design.
By default, the FSBL_PL_LOAD_FROM_OCM_EXCLUDE_VAL value is set to 0 in xfsbl_config.h,
making the bitstream copy and load from DDR in the non-encrypted cases. By seng the
FSBL_PL_LOAD_FROM_OCM_EXCLUDE_VAL value to 1, the user can ensure that bitstream is loaded
from OCM in chunks and not from DDR. Also, if DDR is not present in the design, the bitstream is loaded
from OCM irrespecve of the FSBL_PL_LOAD_FROM_OCM_EXCLUDE_VAL value.
Creating a New Zynq UltraScale+ MPSoC FSBL
Application Project
To create a new Zynq UltraScale+ MPSoC FSBL applicaon in the Vis soware plaorm, do the
following:
1. Click File → New → Applicaon Project.
The New Applicaon Project dialog box appears.
2. In the Project Name eld, type a name for the new project.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 72
3. Select the locaon for the project. To use the default locaon as displayed in the Locaon
eld, leave the Use default locaon check box selected. Otherwise, click to deselect the
check box, then type or browse to the directory locaon.
4. Select Create a new plaorm from hardware (XSA). The Vis IDE lists the all the available
pre-dened hardware designs.
5. Select any one hardware design from the list and click Next.
6. From the CPU drop-down list, select the processor for which you want to build the
applicaon. This is an important step when there are mulple processors in your design. In
this case you can either select psu_cortexa53_0 or psu_cortexr5_0.
7. Select your preferred language: C.
8. Select an OS for the targeted applicaon.
9. Click Next.
10. In the Templates dialog box, select the Zynq UltraScale+ MPSoC FSBL template.
11. Click Finish to create your applicaon project and board support package (if it does not exist).
Phases of FSBL Operation
FSBL operaon includes the following four stages:
• Inializaon
• Boot device inializaon
• Paron loading
• Hando
The following gure shows the stages of FSBL operaon:
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 73
Figure 23: Stages of FSBL
CSU
ROM
1.
Initializatio
n
2. Boot
Device
Initializatio
n
3. Partition
Copy,
Validation
4. Handoff
Applic
ation
wfe
PDR/RST
Handoff to
FSBL
Success
wfe
Error
Error
Error
JTAG
Boot
Error
SLCR RST
(FallBack)
Handof
f
Jtag Boot/
No Image
Partition
Completed
Partition Not
Completed
Fallback
Not Supported
FSBL Stage Diagram
FSBL Block Diagram
1. Initialization
1a. Get Reset Reason
1b. System Initialization
1c. Processor Initialization
1d. DDR Initialization
1e. Board Initialization
1f. Reset Validation
2. Boot Device Installation and
Header Validation
2a. Primary Boot Device
Initialization
2b. Header Validation
2c. Secondary Boot Device
Initialization
XFsbl Initialize()
XFsbl_BootDeviceInitAnd:
Validate()
3. Partition Copy Validation
3a. Partition Header
Validation
3b. Partition Copy
XFsbl_PartitionLoad()
4. Handoff
4a. PM Init
4b. Protection Config
4c. Handoff to CPUs
XFsbl_Handoff()
X19962-101917
Initialization
Inializaon consists of the following four internal stages:
XFsbl_SystemInit
This funcon powers up PL for 1.0 and 2.0 silicon and removes PS-PL isolaon. It inializes
clocks and peripherals as specied in psu-init. This funcon is not called in APU only reset.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 74
Figure 24: FSBL System Initialization
Stage
1a
xFsbl_SystemInit()
System
Initialization
Home
XFsbl_PowerUpIsland()
XFsbl_IsolationRestore()
PowerupPL(Remove Isolation,
For 1.0 & 2.0 silicon)
psu_init()
*DDR, MIO, CLK, PLL Settings,
Peripheral, SERDES Initialization
Configuration
Successful?
no
Error
Lock
Down
yes
In MMUtable for A53 mask
DDR regions “memory”
Stage
1c
X19952-101917
XFsbl_ProcessorInit
Processor inializaon will start in this stage. It will set up the Instrucon and Data caches, L2
caches, MMU sengs, stack pointers in case of A53 and I/D caches, MPU sengs, memory
regions, stack pointers, and TCM sengs for R5-0. Most of these sengs will be performed in
BSP code inializaon. IVT vector is changed to the start of OCM for A53 and to start of TCM
(0x0 in lowvec and 0xffff0000 in highvec) in case of R5-0.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 75
Figure 25: Processor Initialization
Stage
1c
Stage
1b
Partition
Initialization
Home
xFsbl_ProcessorInit()
Handoff from
CSUROM main()
· Configure
Memory
Regions, SP,
TCM (in BSP)
· Enable MPU I/D
Cache (In BSP)
· Copy IVT to
TCM (0x0)
· Change reset
address to TCM
(0x0)
Read Cluster ID
A9?
no
yes
· Configure SP
· Enable MMU, I/D
Cache, L2 Cache
· Change IVT to
start of OCM
XFsbl_RegisterHandlers()
Enable Exceptions
Register Exception Handlers
Stage
1d
X19954-101917
Initialize DDR
DDR would be inialized in this stage. This funcon is not called in Master only reset.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 76
Figure 26: DDR Initialization
Stage
1d
Stage
1c
Stage
1e
DDR Initialization
DDR has ECC?
no
yes
ECC Initialization
of DDR
yes
Higher DDR
Preset!
Stage
1e
no
ECC Initialization
of higher DDR
Stage
1e
X19957-101917
XFsbl_BoardInit
This funcon performs required board specic inializaons. Most importantly, it congures GT
lanes and IIC.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 77
Figure 27: Board Initialization
Stage
1e
Stage
1d
Board Init
no
Stage
1f
Initiate I2C driver
Board is
2cu102 or
2cu100
yes
Stage
1f
Configuration
programming I2C
USB
xxxxx
PCIE Reset for
2CU102
X19960-101917
Boot Device Initialization
XFsbl_PrimaryBootDeviceInit
This stage involves reading boot mode register to idenfy the primary boot device and inialize
the corresponding device. Each boot device driver provides init, copy and release funcons which
are inialized to DevOps funcon pointers in this stage.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 78
Figure 28: Primary Boot Device Initialization
Stage
2a
Stage
1c
xFsbl_PrimaryBootDeviceInit()
yes
Home
Primary Boot Device
Initialization
XFsbl_InitWdt()
XFsbl_CsuDmainit()
Initialize
WDT,CSUDMA
drivers
BootMode==
(JTAG || PJTAG0 ||
PJTAG1)
yes
BootMode ==
(QSP124 || QSP132 || SD0
||EMMC||NAND||SD1 ||
SD1 with level shifter)
no
Wfe;
QSPI ?
no
Error
Lock
Down
yes
Configure QSPI
In Single/Dual/
Stacked mode
Configuration
Success?
no
Error
Lock
Down
NAND ?
yes
Configuration
Success?
no
no
Error
Lock
Down
SD/eMMC
yes
Configuration
Success?
no
no
Create boot
image name
yes
yes
no
no
Stage
2b
X19958-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 79
XFsbl_ValidateHeader
Using the copy funcons provided, the FSBL reads the boot header aributes and image oset
address. It reads the EFUSE bit to check for authencaon. It reads the image header and
validates the image header table. It then reads the Paron Present Device aribute of image
header. A non-zero value indicates a secondary boot device. A zero value indicates that the
secondary boot device is the same as the primary boot device.
Figure 29: Validating Header
Stage
2b
Stage
2a
xFsbl_ValidateHeader()
Home
Header
Validation
XFsbl_ReadIma
geHeader() Read
Image Header
no
Error
Lock
Down
Validate Image
header
Populate handoff
parameter to ATF
Success?
no
Image Header
Table Validation
Successful?
yes
yes
Stage
3a
Image Header Validation
Checks
* No partitions present
* Num of partitions supported
* Start of Partition Header
offset Address check
* Image Header Table
Checksum
X19959-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 80
XFsbl_SecondaryBootDeviceInit
This funcon is called in case of a non-zero value of Paron Present Device aribute of image
header table. It inializes the secondary boot device driver and the secondary boot device would
be used to load all parons by FSBL.
XFsbl_SetATFHandoffParams
TF-A is assumed to be the next loadable paron aer FSBL. It is capable of loading U-Boot and
secure OS and hence, it is passed a hando structure.
The rst paron of an applicaon will have a non-zero execuon address. All the remaining
parons of that applicaon will have 0 as execuon address. Hence look for the non-zero
execuon address for paron which is not the rst one and ensure the CPU is A53.
This funcon sets the hando parameters to the Trusted Firmware-A (TFA). The rst argument is
taken from the FSBL paron header. A pointer to the hando structure containing these
parameters is stored in the PMU_GLOBAL.GLOBAL_GEN_STORAGE6 register, which the TF-A
reads. The structure is lled with magic characters 'X', 'L', 'N', and 'X' followed by the total number
of parons and execuon address of each paron.
Partition Loading
XFsbl_PartitionHeaderValidation
Paron header is validated against various checks. All the required paron variables are
updated at this stage. If the paron owner is not FSBL, paron will be ignored and FSBL will
connue loading the other parons.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 81
Figure 30: Partition Header Validation
Stage
3a
Stage
2b
xFsbl_PartitionHeaderValidation()
Partition Header
Validation
XFsbl_ValidateChecksum()
Validate Partition Header
Checksum ?
Partition
Parsing done
yes
Partition
Owner ==
FSBL
yes
Stage
3b
Home
Stage
4
Update Partition
Variables
Error
Lock
Down
no
no
Stage
3a
Check partition word
lengths for Plain Encrypted
and Autheticated
yes
XFsbl_thecheckvalidMemoryAddress()
Destination Load Address/Eexcuable address Checkt
* No DDR and address in DDR
* Address not in TCM, DRR, PL DDR
Error
Lock
Down
no
yes
Checksum word
offset, Image header
offset, Data word
offset check
yes
* Checksum type not supported
* Destination Cpu not supported
* Running in lockstep mode and destination cpu is r5-0/r5-1
* Destination Cpu same for 2 Images
* XIP Image and length
no
no
no
yes
xFsbl_ValidatePartitionHeader()
X19951-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 82
XFsbl_PartitionCopy
Paron will be copied to the DDR or TCM or OCM or PMU RAM.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 83
Figure 31: Partition Copy
Stage
3b
Stage
3a
xFsbl_PartitionCopy()
yes
Home
Partition
Copy
If no Load address
specified, else
Adress = Load
adress
PS DDR
Present
Skip
Copying(DDRLess)
no
DestinationCPU ==
R5 && Load
address is in
TCM
Destination
Device == PL
no
· Update Load
address Map it
to high TCM
· Power up the
TCMs and
initialize TCM
& CC
yes
Running CPU is
R5 AND Partition is
Non secure Partition
AND Application
address overlaps
with IVT
Copy overlapping
Part of Partition to a
local buffer
yes
Trigger PMU0 IPI
and wait until PMU
Microblaze goes to
sleep before PMU
FW download
Destn Dev =
PMU
Copy
Successful?
yes
Stage
3a
PSOnly Reset
no
yes
Stage
3a
no
no
yes
X19950-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 84
XFsbl_PartitionValidation
Paron will be validated based on the paron aributes. If checksum bit is enabled, then the
paron will be validated rst for checksum correctness and then, based on the authencaon
ag, it would be authencated. If encrypon ag is set, then the paron will be decrypted and
then copied to the desnaon.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 85
Figure 32: Partition Validation Function
Xsecure_AesDecrypt()
Decrypt Image from DDR/
TCMP/PMU_RAM
To DDR/TCM/PMU_RAM
Stage
3c
Stage
3b
Checksum
enabled?
Stage xFsbl_PartitionValidation()
Checksum
Validation
Sucessful?
yes no
Home
Authenticaion
Enabled?
yes no
XFabl_Authentication()
Authentication Validation
Successful?
yes
no
Partition
Validation
(PS Image II
PMU firmware) &&
(Encrypted) ?
yes
no
yes
Decryption
Successful ?
Error
Xxx
xxxx
no
FsblHookBefore
BitstreamDownload
yes
Initialize PCAP
Interface
For 3.0 Ps version &
above Powerup PL
Encrypted
bitstream?
Xsecure_AesDecrypt()
Send bitstream via
CSUDMA_AES_to PCAP
yes
Send bitstream to
PCAP through
CSUDMA
PL Done
Successful?
Wait until PL Done
no
Error
Xxx
xxxx
Remove the Isolation
for PS-PL
Provide PS-PL
reset
FSBL Hook after
Bitstream download
Stage
4
PL Image?
yes
no
no
X19949-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 86
Handoff
In this stage, protection_config funcons from psu_init will be executed and then, any
hando funconality is executed. Also PS-PL isolaon is removed uncondionally. R5 will be
brought out of reset if there is any paron supposed to run on its cores. R5-0/R5-1 will be
congured to boot in lowvec mode or highvec mode as per the sengs provided by you while
building the boot image. The hando address in lowvec mode is 0x0 and 0xffff0000 in
highvec mode. Lowvec/Highvec informaon should be specied by you while building the boot
image. Aer all the other PS images are done, then the running CPU will be handed o with an
update of the PC value. If there is no image to hand o for the running the CPU, FSBL will be in
wfe loop.
Any running processor cannot pass any parameters to any other processor. Any communicaon
between various parons can happen by reading from (or wring to) the PMU global registers.
Hando on the running processor involves updang Program Counter (PC) of the running
processor, as is done in the case of APU Reset. Hando to other processors involves updang
their PCs and bringing the processors out of reset.
XFsbl_PmInit
This funcon inializes and congures the Inter Processor Interrupts (IPI). It then writes the PM
conguraon object address to an IPI buer and triggers an IPI to the target. The PMU rmware
then reads and congures the device nodes as specied in the conguraon object.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 87
Figure 33: PM Initialization
4a
Stage
3c
PM_INIT
Is PS-only
Reset
Remove PS-PL
isolation
Is APV-only
Reset
PM Init
4b
4c
yes
no
no
yes
X19946-101917
Protection Configuration
In this stage, protection_config funcons from psu_init will be executed. The applicaon
of protecon happens in this stage.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 88
Figure 34: Protection Configuration
Disable tamper
responses
4b
4a
Protection Config
Is protection
byte met
psu-apply xxxxx3
psu-ocm-protection
4c
Stage
psu-protection
psu-protection mode
Enable tamper
response
yes
no
X19947-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 89
Handoff
Hando on the running processor involves updang Program Counter (PC) of the running
processor, as is done in the case of APU Reset. Hando to other processors involves updang
their PCs and bringing the processors out of reset. A53 FSBL will bring R5 out of reset if there is
any paron to run on it. R5 will be congured to boot in lowvec mode or highvec mode as per
the sengs provided by you while building the boot image. The hando address in lowvec mode
is 0x0 and 0xffff0000 in highvec mode.
You must specify Lowvec/Highvec informaon while building the boot image. Aer all the other
PS images are done, then running the CPU image will be handed o to that cpu with an update
on the PC value. If there no image for the running CPU, it will be in wfe loop.
Figure 35: Handoff
Stage
4c
Stage
4b
xFsbl_Handoff()
Home
Handoff
Copy Arm
predefined code
to 0xffff0000
PS Image present?
yes
· Update Reset
vector address
at 0xFFFFFF00
· Take CPU out of
reset
Running CPU
handoff image
present ?
no
wfe
Disable
exceptions
yes
R5 ?
no
Update PC
yes
HANDOFF
· Copy the original
vector table to
TCM if required
· Update PC
X19948-101917
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 90
Supported Handoffs
The following table shows the various combinaons of handos that are supported in FSBL.
Table 17: Supported Handoffs
FSBL Application Processor Cores Execution Address
64-bit 64-bit All (i.e. A53-0, A53-1, A53-2, A53-3) Any Address
64-bit 32-bit A53-1, A53-2, A53-3 0x0
32-bit 32-bit A53-0 Any Address
32-bit 32-bit A53-1, A53-2, A53-3 0x0
32-bit 64-bit A53-1, A53-2, A53-3 Any Address
Error Lock Down
XFsbl_ErrorLockDown funcon handles errors in FSBL. This funcon is called whenever the
return value of a funcon is unsuccessful. This funcon updates error status register and then
loops indenitely, if fallback is not supported.
In case the boot mode supports fallback, MulBoot oset register is updated and then waits for a
WDT reset to occur. On reboot, bootROM and FSBL read the image from the new address
calculated from MulBoot oset, thus loading a new image.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 91
Figure 36: Error Lock Down Function
XFsbl_ErrorLockDown()
Home
Update Error
Status Register
BootMode ==
( QSP124 || QSP132 || SD0
||EMMC|NAND|SD1 ||
SD1 with level shifter)
no
yes
Fallback not
supported by this
bootmode
XFsbl_HookBefor
eFallback() FSBL
Hook Before
Fallback
yes
Update Multiboot
Address register
wfe
SLCR Reset
X19953-101917
Miscellaneous Functions
The following funcons are available in FSBL:
XFsbl_PrintArray
This funcon prints enre array in bytes as specied by the debug type.
void XFsbl_PrintArray (u32 DebugType, const u8 Buf[], u32 Len, const char
*Str);
Table 18: XFsbl_PrintArray Parameters in FSBL
Parameters Description
DebugType Printing of the array is performed as defined by the debug
type.
Buf Pointer to the buffer to be printed
Len Length of the bytes to be printed
Str Pointer to the data that is printed
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 92
XFsbl_Strcpy
This funcon to copy the source string to the desnaon string.
char *XFsbl_Strcpy(char *DestPtr, const char *SrcPtr)
Table 19: XFsbl_Strcpy Parameters in FSBL
Parameters Description
DestPtr Pointer to the buffer to be printed
SrcPtr Pointer to the buffer containing the source string
XFsbl_Strcat
This funcon to append the second string to the rst string.
char* XFsbl_Strcat(char* Str1Ptr, const char* Str2Ptr)
Table 20: XFsbl_Strcat Parameters in FSBL
Parameters Description
Str1Ptr Pointer to the original string to which string pointed to by
Str2Ptr would be appended
Str2Ptr Pointer to the second string
XFsbl_Strcmp
This funcon compares strings.
s32 XFsbl_Strcmp( const char* Str1Ptr, const char* Str2Ptr)
Table 21: XFsbl_Strcmp Parameters in FSBL
Parameters Description
Str1Ptr Pointer to the first string
Str2Ptr Pointer to the second string
XFsbl_MemCpy
This funcon copies the memory contents pointed to by SrcPtr to the memory pointed to by
DestPtr. Len is number of bytes to be copied.
void* XFsbl_MemCpy(void * DestPtr, const void * SrcPtr, u32 Len)
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 93
Table 22: XFsbl_MemCpy Parameters in FSBL
Parameters Description
SrcPtr Pointer to the memory contents to be copied
DestPtr Pointer to the destination
Len Length of the bytes to be printed
XFsbl_PowerUpIsland
This funcon checks the power state of one or more power islands and powers them up if
required.
u32 XFsbl_PowerUpIsland(u32 PwrIslandMask)
Table 23: XFsbl_PowerUpIsland Parameters in FSBL
Parameters Description
PwrIslandMask Mask of island that needs to be powered up
XFsbl_IsolationRestore
This funcon requests isolaon restore through the PMU rmware.
u32 XFsbl_IsolationRestore(u32 IsolationMask);
Table 24: XFsbl_IsolationRestore Parameters in FSBL
Parameters Description
IsolationMask Mask of the entries for which isolation is to be restored
XFsbl_SetTlbAttributes
This funcon sets the memory aributes for a secon in the translaon table.
void XFsbl_SetTlbAttributes(INTPTR Addr, UINTPTR attrib);
Table 25: XFsbl_SetTlbAttributes Parameters in FSBL
Parameters Description
Addr Address for which the attributes are to be set
Attrib Attributes for the memory region
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 94
XFsbl_GetSiliconIdName
This funcon reads the CSU_ID_CODE register and calculates the SvdId of the device. It returns
the corresponding deviceID name.
const char *XFsbl_GetSiliconIdName(void);
XFsbl_GetProcEng
This funcon determines and returns the engine type. Currently only CG, EG, and EV engine
types are supported.
const char *XFsbl_GetProcEng(void);
XFsbl_CheckSupportedCpu
This funcon checks if a given CPU is supported by this variant of Silicon. Currently it checks if it
is CG part and disallows hando to A53_2/3 cores.
u32 XFsbl_CheckSupportedCpu(u32 CpuId);
Table 26: XFsbl_CheckSupportedCpu Parameters in FSBL
Parameters Description
Cpuld Checks if the processor is A53_2 or A53_3 or not.
XFsbl_AdmaCopy
This funcon copies data memory to memory using ADMA. You must take care of cache
invalidaon and ushing. ADMA also should be congured to simple DMA before calling this
funcon.
u32 XFsbl_AdmaCopy(void * DestPtr, void * SrcPtr, u32 Size);
Table 27: XFsbl_AdmaCopy Parameters in FSBL
Parameters Description
DestPtr Pointer to the destination buffer to which data needs to be
copied
SrcPtr Pointer to the source buffer from which data needs to be
copied
Size Number of bytes of data that needs to be copied
XFsbl_GetDrvNumSD
This funcon is used to obtain drive number based on design and boot mode.
u32 XFsbl_GetDrvNumSD(u32 DeviceFlags);
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 95
Table 28: XFsbl_GetDrvNumSD Parameters in FSBL
Parameters Description
Device flags Contains the boot mode information, that is, one of SD0,
SD1, eMMC, or SD1-LS boot modes
XFsbl_MakeSdFileName
This funcon returns the le name of the boot image. The name is deduced from the parameters.
void XFsbl_MakeSdFileName(char*XFsbl_SdEmmcFileName, u32 MultiBootReg, u32
DrvNum);
Table 29: XFsbl_MakeSdFileName Parameters in FSBL
Parameters Description
XFsbl_SdEmmcFileName Contains the final file name
Multiboot reg The value of the MultiBoot register gets appended to the file
name, if its value is non zero
DrvNum Differentiates between SD0 and SD1 logical drives
Hooks in FSBL
Hooks are the funcons that can be dened by you. FSBL provides blank funcons and executes
them from certain strategic locaons. The following table shows the currently available hooks.
Table 30: Hooks in FSBL
Hook Purpose/Location Hook Function Name
Before PL bitstream loading XFsbl_HookBeforeBSDownload()
After PL bitstream loading XFsbl_HookAfterBSDownload()
Before (the first) Handoff (to any application) XFsbl_HookBeforeHandoff()
Before fallback XFsbl_HookBeforeFallback()
To add more initialization code, in addition to that in psu_init
or to replace psu_init with custom initialization
XFsbl_HookPsuInit(()
See FSBL wiki page for more informaon on FSBL.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 96
Using the Ethernet-Based Recovery Tool
The SOM soluon stack includes an Ethernet-based recovery mode that can be used with an
applicaon on a host PC to provide a full factory QSPI image reset/reload mechanism. The
recovery tool will be maintained in a reserved secon of the QSPI memory. Ethernet-based
recovery was selected over USB as many security minded customers prefer to disable the USB
but are using Ethernet in their applicaons. The Ethernet recovery applicaon is an applicaon
that runs a simplied Ethernet stack for interacng with a customer/engineer’s host machine (for
example, a laptop) to allow a manual update/overwrite of Image A and Image B on QSPI on SOM.
Ethernet-based recovery requires no incremental tools on your PC besides a web browser. Your
PC must have the new rmware binary le in its local le-system. You should be able to handle
downloading the associated Xilinx factory boot le updates from a Xilinx repository, and the on-
target rmware will handle the mapping of a given le to physical address and the act of wring
to ash. Image recovery ulity uses a xed IP address of 192.168.0.111.
The Ethernet-based recovery tool is capable of reading the sideband control EEPROMs to verify
the make and model of the SOM. The Ethernet factory fallback/recovery process is iniated by
holding the FWUEN buon during a power on reset sequence. The tool requires the rmware
update buon to be depressed to over-write any rmware contents. It updates the
corresponding A/B persistent registers to an appropriate state.
To use the Ethernet-based recovery tool, follow these steps:
1. Hold the rmware update enable (FWUEN) buon when powering on the device.
2. Connect the PC to the KV260 Starter Kit via Ethernet as shown in the following gure.
3. Set the PC to a stac IP that is on the same subnet as the recovery tool (192.168.0.xyz), but
not the same IP address.
4. Use a web-browser (for example, Chrome or Firefox) to enter the URL hp://192.168.0.111
for access to the Ethernet-based recovery tool.
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 97
5. Use the GUI to update either the A or B boot rmware parons with a boot.bin le from
the le system on the PC.
The Ethernet recovery tool interface is shown in the following gure.
Figure 37: Ethernet Recovery Tool
Chapter 7: System Boot and Configuration
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 98
Chapter 8
Security Features
This chapter details the Zynq
®
UltraScale+™ MPSoC features that you can leverage to address
security during boot me and run me of an applicaon. The Secure Boot mechanism is
described in detail in this link to the Security chapter of the Zynq UltraScale+ Device Technical
Reference Manual (UG1085).
The system protecon units (SPU) provide the following hardware features for run-me security
of an applicaon running on Zynq UltraScale+ MPSoCs:
• Xilinx Memory Protecon Unit
• Xilinx Peripheral Protecon Unit
• System Memory Management Unit
• A53 Memory Management Unit
• R5 Memory Protecon Unit
One of the runme security features is access controls on the PMU and CSU global registers
from Linux. These registers are classied into two lists: The white list (accessible all the me by
default) and the black list (accessible only when a compile me ag is set). For more details, see
CSU/PMU Register Access.
Boot Time Security
This secon details the various boot image formats for authencaon and encrypon.
IMPORTANT!
For Zynq MPSoC, when RSA_EN eFUSE is not programmed and
BOOT.BIN
  does not have
BH_AUTH enabled, FSBL can load bin as non-secure even if the parons are authencated. It is a new
feature in 2021.1, which is disabled by default. To enable it, set
FSBL_UNPROVISIONED_AUTH_SIGN_EXCLUDE_VAL to 0 in xfsbl_cong.h
Encryption
Zynq UltraScale+ MPSoCs has a 256-bit AES-GCM hardware engine that supports condenality
of your boot images, and can also be used post-boot to encrypt and decrypt data.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 99
The AES cryptographic engine has access to a diverse set of key sources. For more informaon
on the key sources, see Zynq UltraScale+ Device Technical Reference Manual (UG1085).
The red key is used to encrypt the image. During the generaon of the boot le (BOOT.bin), the
red key, and the inializaon vector (IV) must be provided to the Bootgen tool in .nky le
format.
PMU rmware can be loaded by CSU bootROM or FSBL.
IMPORTANT! If both the FSBL and PMU rmware are encrypted, the PMU rmware must be loaded by
the FSBL (and not the CSU bootROM) to avoid reusing the AES Key/IV pair. For more informaon, see
Xilinx Answer 70622.
The following BIF le is an example encrypted image, where PMU rmware is loaded by FSBL:
the_ROM_image:
{
[aeskeyfile] bbram.nky [keysrc_encryption] bbram_red_key
[bootloader, encryption=aes, destination_cpu=a53-0] ZynqMP_Fsbl.elf
[destination_cpu = pmu, encryption=aes] pmufw.elf
}
BIF File with BBRAM Red Key
The following BIF le sample shows the red key stored in BBRAM:
the_ROM_image: { [aeskeyfile] bbram.nky
[keysrc_encryption] bbram_red_key
[bootloader, encryption=aes, destination_cpu=a53-0] ZynqMP_Fsbl.elf
[destination_cpu = a53-0, encryption=aes] App_A53_0.elf
}
BIF File with eFUSE Red Key
The following BIF le sample shows the red key stored in eFUSE.
the_ROM_image: { [aeskeyfile] efuse.nky
[keysrc_encryption] efuse_red_key
[bootloader, encryption=aes, destination_cpu=a53-0] ZynqMP_Fsbl.elf
[destination_cpu = a53-0, encryption=aes] App_A53_0.elf
}
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 100
BIF File with an Operational Key
For creang a boot image using Bootgen with an operaonal key (op key), you must provide the
tool with the operaonal key, along with the red key and IV in an .nky le. Bootgen places this
operaonal key in a header and encrypts it with the device red key. The result is what is called an
encrypted secure header. The main advantage of this is that it minimizes the use of the device
key, thus liming its exposure. For more details, refer to “Minimizing Use of the AES Boot Key
(OP Key Opon)” in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
the_ROM_image:
{
[aeskeyfile] bbram.nky [fsbl_config] opt_key [keysrc_encryption]
bbram_red_key
[bootloader, encryption=aes, destination_cpu=a53-0] ZynqMP_Fsbl.elf
[destination_cpu = a53-0, encryption=aes] App_A53_0.elf
}
Using Op Key to Protect the Device Key in a Development Environment
The following steps provide a soluon in a scenario where two development teams Team-A
(secure team), which manages the secret red key and Team-B (non-secure team) work
collaboravely to build an encrypted image without sharing the secret red key. Team-A manages
the secret red key. Team-B builds encrypted images for development and test. However, it does
not have access to the secret red key.
Team-A encrypts the boot loader with the device key (using the Op Key opon) and delivers the
encrypted bootloader to Team-B. Team-B encrypts all the other parons using the Op Key.
Team-B takes the encrypted parons that they created and the encrypted boot loader they
received from the Team-A and uses Bootgen to ‘stch’ everything together into a single
boot.bin.
The following procedures describe the steps to build an image:
Procedure 1
In the inial step, Team-A encrypts the boot loader with the device key using the opt_key opon,
delivers the encrypted boot loader to Team-B. Now, Team-B can create the complete image at a
go with all the parons and the encrypted boot loader using the operaonal key as device key.
1. Encrypt boot loader with device key:
bootgen -arch zynqmp -image stage1.bif -o fsbl_e.bin -w on -log error
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 101
Example stage1.bif:
stage1:
{
[aeskeyfile] aes.nky
[fsbl_config] opt_key
[keysrc_encryption] bbram_red_key
[bootloader,destination_cpu=a53-0,encryption=aes]fsbl.elf
}
Example aes.nky for stage1:
Device xc7z020clg484;
Key 0 AD00C023E238AC9039EA984D49AA8C819456A98C124AE890ACEF002100128932;
IV 0 F7F8FDE08674A28DC6ED8E37;
Key Opt 229C993D1310DD27B6713749B6D07FCF8D3DCA01EC9C64778CBAF457D613508F;
2. Aach the encrypted boot loader and rest of the parons with the operaonal key as device
key to form a complete image:
bootgen -arch zynqmp -image stage2a.bif -o final.bin -w on -log error
Example of stage2.bif:
stage2:
{
[aeskeyfile] aes-opt.nky
[bootimage]fsbl_e.bin
[destination_cpu=a53-0,encryption=aes]hello.elf
[destination_cpu=a53-1,encryption=aes]hello1.elf
}
Example aes-opt.nky for stage2:
Device xc7z020clg484;
Key 0 229C993D1310DD27B6713749B6D07FCF8D3DCA01EC9C64778CBAF457D613508F;
IV 0 F7F8FDE08674A28DC6ED8E37;
Procedure 2
In the inial step, Team-A encrypts the boot loader with the device key using the opt_key opon
and delivers the encrypted boot loader to Team-B. Now, Team-B can create encrypted images for
each paron independently, using the operaonal key as the device key. Finally, Team-B can use
Bootgen to stch all the encrypted parons and the encrypted boot loader, to get the complete
image.
1. Encrypt boot loader with device key:
bootgen -arch zynqmp -image stage1.bif -o fsbl_e.bin -w on -log error
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 102
Example stage1.bif:
stage1:
{
[aeskeyfile] aes.nky
[fsbl_config] opt_key
[keysrc_encryption] bbram_red_key
[bootloader,destination_cpu=a53-0,encryption=aes]fsbl.elf
}
Example aes.nky for stage1:
Device xc7z020clg484;
Key 0 AD00C023E238AC9039EA984D49AA8C819456A98C124AE890ACEF002100128932;
IV 0 F7F8FDE08674A28DC6ED8E37;
Key Opt 229C993D1310DD27B6713749B6D07FCF8D3DCA01EC9C64778CBAF457D613508F;
2. Encrypt the rest of the parons with operaonal key as device key:
bootgen -arch zynqmp -image stage2a.bif -o hello_e.bin -w on -log error
Example of stage2a.bif:
stage2a:
{
[aeskeyfile] aes-opt.nky
[destination_cpu=a53-0,encryption=aes]hello.elf
}
bootgen -arch zynqmp -image stage2b.bif -o hello1_e.bin -w on -log error
Example of stage2b.bif:
stage2b:
{
[aeskeyfile] aes-opt.nky
[destination_cpu=a53-1,encryption=aes]hello1.elf
Example of aes-opt.nky for stage2a and stage2b:
Device xc7z020clg484;
Key 0 229C993D1310DD27B6713749B6D07FCF8D3DCA01EC9C64778CBAF457D613508F;
IV 0 F7F8FDE08674A28DC6ED8E37;
3. Use Bootgen to stch the above to form a complete image:
Example of stage3.bif:
stage3:
{
[bootimage]fsbl_e.bin [bootimage]hello_e.bin [bootimage]hello1_e.bin
}
Note: Key Opt of aes.nky is same as Key 0 in aes-opt.nky and IV 0 must be same in both nky les.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 103
BIF File for Black Key Stored in eFUSE
For customers who would like to have the device key stored encrypted when not in use, the
physical unclonable funcon (PUF) can be used. Here, the actual red key is encrypted with the
PUF key encrypon key (KEK), which is an encrypon key that is generated by the PUF. The
device will decrypt the black key to get the actual red key, so you need to provide the KEK details
in BIF, such as shuer value, KEK IV to Bootgen. The black key can be stored in either eFUSE or
the Boot Header. Shuer value indicates the me for which the oscillator values can be captured
for PUF. This value must always be 0x100005E.
For more details, refer to “Storing Keys in Encrypted Form (Black)” in the Zynq UltraScale+ Device
Technical Reference Manual (UG1085).
The following BIF example shows storage of the black key in eFUSE.
the_ROM_image:
{
[pskfile]PSK.pem
[sskfile]SSK.pem
[aeskeyfile]red.nky
[keysrc_encryption] efuse_blk_key
[fsbl_config] shutter=0x0100005E
[auth_params] ppk_select=0
[bootloader, encryption = aes, authentication = rsa,
destination_cpu=a53-0]fsbl.elf
[bh_key_iv] black_key_iv.txt
}
BIF File for Black Key Stored in Boot Header
The following BIF le sample shows boot header black key encrypon:
the_ROM_image:
{
[aeskeyfile] redkey.nky
[keysrc_encryption] bh_blk_key
[bh_keyfile] blackkey.txt
[bh_key_iv] black_key_iv.txt
[fsbl_config] pufhd_bh , puf4kmode , shutter=0x0100005E, bh_auth_enable
[pskfile] PSK.pem
[sskfile] SSK.pem
[bootloader,authentication=rsa , encryption=aes,
destination_cpu=a53-0]fsbl.elf
[puf_file]hlprdata4k.txt
}
Note: Authencaon of boot image is compulsory for using black key encrypon.
To generate or program the eFUSEs with the back key, see Programming BBRAM and eFUSEs
(XAPP1319).
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 104
BIF File for Obfuscated Form (Gray) Key Stored in eFUSE
If you would like to have the device key store in obfuscated form, you can encrypt the actual red
key with the family key which is an encrypon key. Device will decrypt the obfuscated key to get
the actual red key. Hence, you need to provide the required inputs to Bootgen. The obfuscated
key can be stored in either eFUSE or the Boot Header.
For more details, see Storing Keys in Obfuscated Form (Gray) secon in the Zynq UltraScale+
Device Technical Reference Manual (UG1085).
Note: The family key is the same for all devices within a given Zynq UltraScale+ MPSoCs family. This
soluon allows you to distribute the obfuscated key to contract manufacturer's without disclosing the
actual user key.
The following example shows storage of the obfuscated key in eFUSE:
the_ROM_image:
{
[aeskeyfile] red.nky
[keysrc_encryption] efuse_gry_key
[bh_key_iv] bhkeyiv.txt
[bootloader, encryption=aes, destination_cpu=a53-0] fsbl.elf
}
The following example shows storage of the obfuscated form (gray) key in boot header:
the_ROM_image:
{
[aeskeyfile] red.nky [keysrc_encryption] bh_gry_key [bh_key_iv]
bhkeyiv.txt
[bh_keyfile] bhkey.txt
[bootloader, encryption=aes, destination_cpu=a53-0] fsbl.elf
}
To Generate Obfuscated Key with Family Key:
Use Xilinx tools (Bootgen) to create the Obfuscated key. However, the family key is not
distributed with the Xilinx development tools. It is provided separately. The family key received
from Xilinx should be provided in the bif as shown in the example below.
IMPORTANT!
To receive the family key, please contact secure.solu[email protected].
Sample bif to generate Obfuscated key:
all:
{
[aeskeyfile] aes.nky
[familykey] familyKey.cfg
[bh_key_iv] bhiv.txt
}
For more informaon, see Bootgen User Guide (UG1283).
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 105
Using Bootgen to Generate Keys
If you are using Bootgen to create keys, NIST approved KDF is used, which is Counter Mode KDF
with CMAC as the PRF.
With a Single Key/IV pair:
• If seed is specied - Key Generaon is based on Seed.
• If seed is NOT specied - Key Generaon is based on Key0.
IMPORTANT! For producon devices, make sure that the Seed or Key0 has been generated by a
cryptographically strong generator such as a true random number generator.
If an empty le is menoned, Bootgen generates a seed with me based randomizaon. This is
not a standard like the KDF. This seed will in turn be the input for KDF to generate the Key/IV
pairs.
BIF File with Multiple AESKEY Files
The following BIF le samples show the encrypons using aeskey les:
One AES Key / Partition
You may specify mulple .nky les, one for each paron in the image. The parons are
encrypted using the key that is specied before the paron.
sample_bif:
{
[aeskeyfile] test1.nky
[bootloader, encryption=aes] fsbl.elf
[aeskeyfile] test2.nky
[encryption=aes] hello.elf
[aeskeyfile] test3.nky
[encryption=aes] app.elf
}
The fsbl.elf paron is encrypted using the keys from test1.nky le. If you assume that
the hello.elf le has two parons since it has two loadable secons, then both the
parons are encrypted using keys from test2.nky le. The app.elf paron is encrypted
using keys from test3.nky le.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 106
One AES Key / Each Partition (Multiple Loadable Sections Scenario)
You may specify mulple .nky les, one for each paron in the image. The parons are
encrypted using the key that is specied before the paron. You are allowed to have unique key
les for each of the paron created due to mulple loadable secons by having key le names
appended with ‘.1’, ‘.2’...’.n’ in the same directory of the key le meant for that paron.
sample_bif:
{
[aeskeyfile] test1.nky
[bootloader, encryption=aes] fsbl.elf
[aeskeyfile] test2.nky
[encryption=aes] hello.elf
[aeskeyfile] test3.nky
[encryption=aes] app.elf
}
The fsbl.elf paron is encrypted using the keys from test1.nky le. Assume that the
hello.elf le has three parons since it has three loadable secons, and hello.elf.0 is
encrypted using the keys from test2.nky le, hello.elf.1 is encrypted using the keys from
test2.1.nky, and hello.elf.2 is encrypted using the keys from test2.2.nky le. The
app.elf paron is encrypted using keys from test3.nky le.
Using the same .nky le across mulple parons, reuses the AES Key and AES Key/IV Pair in
each paron. Using the AES key across mulple parons increases the exposure of the key
and violates NIST. 800-38D. To avoid the re-use of AES Key/IV pair, Bootgen increments the IV
with the paron number. To avoid the re-use of both AES Key and AES Key/IV pair, Bootgen
allows you to provide mulple .nky les, one for each paron.
IMPORTANT!
To avoid key re-use, support for single nky le across mulple parons will be deprecated.
CAUTION! Using a single
.nky
  le with mulple parons means that the same key is being used in
each paron, which violates NIST. 800-38D. A warning is issued in the current release with the plan to
generate an error in future releases.
Note: Key0/IV0 - should be the same in all the nky les.
If you specify mulple keys and if the number of keys are less than the number of blocks to be
encrypted, the Bootgen is ERRORED OUT.
If you need to specify mulple Key/IV pairs, you must specify (number-of-blocks+1) pairs. The
extra Key/IV pair is for SH. Ex: If blocks=4;8;16 - you have to specify 4+1=5 Key/IV pairs.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 107
Authentication
The SHA hardware accelerator included in the Zynq UltraScale+ MPSoC implements the SHA-3
algorithm and produces a 384-bit digest. It is used together with the RSA-4096 accelerator to
provide image authencaon These blocks (SHA-3/384, and RSA) are hardened and part of
crypto interface block (CIB). You can use authencaon by itself or in conjuncon with
encrypon.
Authencaon ow treats the FSBL as raw data, where it makes no dierence whether the
image is encrypted or not. There are two level of keys: primary key (PK) and secondary Key (SK).
Each key has two complementary parts: secret key and public key:
• PK contains primary public key (PPK) and primary secret key (PSK).
• SK contains secondary public key (SPK) and secondary secret key (SSK).
The hardened RSA block in the CIB is a Montgomery mulplier for acceleraon of the math
required for RSA. The hardware accelerator can be used for signature generaon or vericaon.
The ROM code only supports signature vericaon. Secret keys are only used in the signature
generaon stage when the cercate is generated by Bootgen.
IMPORTANT!
Signature generaon is not done on the device, but in soware during preparaon of the
boot image.
To beer understand the format of the authencaon cercate, see Bootgen User Guide
(UG1283).
As with all asymmetric algorithms, the private (secret) keys (PSK and SSK) are used to sign while
the public versions (PPK and SPK) are used to verify (authencate). The equaons for each
signature (SPK, boot header, and boot image) are listed here:
• SPK signature. The 512 bytes of the SPK signature is generated by the following calculaon:
SPK signature = RSA(PSK, padding || SHA(SPK+ auth_header)).
• Boot header signature. The 512 bytes of the boot header signature is generated by the
following calculaon:
Boot header signature = RSA(SSK, padding || SHA(boot header)).
• Boot image signature. The 512 bytes of the boot image signature is generated by the
following calculaon:
BI signature = RSA(SSK, padding || SHA(PFW + FSBL + authentication
certificate)).
Note: For SHA-3 authencaon, Xilinx uses Keccak SHA3 to calculate hash on boot header, PPK hash and
boot image. NIST-SHA3 is used for all other parons which are not loaded by ROM. The dierence
between the two is padding (10*1 vs 0110*1). Request XPT475 from your Xilinx representave for details
on the NIST variances.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 108
Bootgen supports RSA-4096 signature generaon only. The modulus, exponenaon and
precalculated R^2 Mod N are required. Soware is supported only for RSA public key encrypon,
for encrypng the signature RSA engine requires modulus, exponenaon and pre-calculated
R^2 Mod N, all these are extracted from keys.
BIF File with SHA-3 Boot Header Authentication and PPK0
The following BIF le sample supports the BH RSA opon. This opon supports integraon and
test prior to the system being elded. For more details, see “Integraon and Test Support (BH
RSA Opon)” in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
The BIF le is for SHA-3 boot header authencaon, where actual PPK hash is not compared
with the eFUSE stored value.
the_ROM_image: {
[fsbl_config] bh_auth_enable
[auth_params] ppk_select=0; spk_id=0x00000000
[pskfile] primary_4096.pem
[sskfile] secondary_4096.pem
[bootloader, authentication=rsa, destination_cpu=a53-0] fsbl.elf
[pmufw_image, authentication=rsa] xpfw.elf
}
BIF File with SHA-3 eFUSE RSA Authentication and PPK0
The following BIF le sample shows eFUSE RSA authencaon using PPK0 and SHA-3.
the_ROM_image:
{
[auth_params]ppk_select=0;spk_id=0x00000001
[pskfile]psk.pem
[sskfile]ssk.pem
[bootloader, authentication = rsa, destination_cpu=a53-0]zynqmp_fsbl.elf
[destination_cpu = a53-0, authentication = rsa]Application.elf
}
Enhanced RSA Key Revocation Support
The RSA key provides the ability to revoke the secondary keys of one paron without revoking
them for all parons.
Note: Primary key should be the same across all parons.
This is achieved by using USER_FUSE0 to USER_FUSE7 eFuses (one can revoke up to 256 keys, if
all are not required for their usage) with the new BIF parameter spk_select.
The following BIF le sample shows enhanced user fuse revocaon:
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 109
Image header and FSBL uses dierent SSK's for authencaon (ssk1.pem and ssk2.pem
respecvely) with the following bif input.
the_ROM_image: {
[auth_params]ppk_select = 0
[pskfile]psk.pem
[sskfile]ssk1.pem
[bootloader, authentication = rsa, spk_select = spk-efuse, spk_id =
x00000001, sskfile = ssk2.pem]zynqmp_fsbl.elf
[destination_cpu =a53-0, authentication = rsa, spk_select = user-
efuse,spk_id = 0x1, sskfile = ssk3.pem]Application1.elf
[destination_cpu =a53-0, authentication = rsa, spk_select = spk-efuse,
spk_id = 0x00000001, sskfile = ssk4.pem]Application2.elf
}
Same SSK will be used for both Image header and FSBL (ssk2.pem), if separate SSK is not
menoned.
the_ROM_image: {
[auth_params]ppk_select = 0 [pskfile]psk.pem
[bootloader, authentication = rsa, spk_select = spk-efuse, spk_id =
0x00000001, sskfile = ssk2.pem]zynqmp_fsbl.elf
[destination_cpu =a53-0, authentication = rsa, spk_select = user-efuse,
spk_id = 1, sskfile = ssk3.pem]Application1.elf
[destination_cpu =a53-0, authentication = rsa, spk_select = spk-efuse,
spk_id = 0x00000001, sskfile = ssk4.pem]Application2.elf
}
spk_select = spk-efuse indicates that spk_id eFuse will be used for that paron.
spk_select = user-efuse indicates that user eFuse will be used for that paron.
Parons loaded by CSU ROM will always use spk_efuse.
Note: The spk_id eFuse species which key is valid. Hence, the ROM checks the enre eld of spk_id
eFuse against the SPK ID to make sure it is a bit for bit match.
Valid range of spk_id for spk_select user-efuse is 0x1 to 0x100 (in decimal 1 to 256). The
user eFuse species which key ID is not valid (has been revoked). Hence, the rmware (non-
ROM) checks to see if a given user eFuse that represents the SPK ID has been programmed.
Bitstream Authentication Using External
Memory
Authencaon of a bitstream is dierent from all other parons. The FSBL can be wholly
contained within the OCM, and therefore authencated and decrypted inside of the device. For
the bitstream, the size of the le is so large that it cannot be wholly contained inside the device
and external memory must be used. The use of external memory creates a challenge to maintain
security because an adversary may have access to this external memory.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 110
The following secon describes how the bitstream is authencated securely using external
memory.
Bootgen
When bitstream is requested for authencaon, Bootgen divides the whole bitstream into 8 MB
blocks and has an authencaon cercate for each block.
If a bitstream is not in mulples of 8 MB, the last block contains the remaining bitstream data.
Figure 38: Bitstream Blocks
Boot Header
FSBL
FSBL AC
8MB block 1
8MB block 2
.
PL Bitstream Data
.
.
LastBlock(remaining)
Block1 AC
Block2 AC
.
.
.
Last Block AC
Whole
Bitstream
Authentication
certificates of
bitstream
X19220-110320
When authencaon and encrypon are both enabled, encrypon is rst done on the bitstream.
Then Bootgen divides the encrypted data into blocks and places an authencaon cercate for
each block.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 111
Software
To securely authencate the bitstream paron, FSBL uses the TF-A secon's OCM memory to
copy the bitstream in chunks from FLASH or DDR.
IMPORTANT! While creang a boot image, the bitstream paron should be before the TF-A paron.
Otherwise, TF-A memory is over-wrien while processing the bitstream paron.
The workow for the DDR and DDR-less systems is nearly idencal. The only dierence is that
for systems with the DDR, FSBL copies the enre bitstream paron (bitstream and
authencaon cercates) to the DDR from the FLASH devices, because DDR is faster to
access. FSBL then, each me, copies a chunk of bitstream from the DDR. For the DDR-less
systems, FSBL copies a chunk of bitstream directly from the FLASH devices.
The following is the soware workow for authencang the bitstream:
1. FSBL idenes the availability of the DDR on the system based on the XFSBL_PS_DDR
macro. FSBL has two buers in OCM, ReadBuer buer of size 56 KB and HashsOfChunks[]
to store intermediate hashs calculated for each 56 KB of 8 MB blocks.
2. FSBL copies a 56 KB chunk from the rst 8 MB block to ReadBuer.
3. FSBL calculates hash on 56 KB and stores in HashsOfChunks.
4. FSBL repeats the previous steps unl the enre 8 MB of block is completed.
Note: 56 KB is taken for performance; it can be of any size.
5. FSBL authencates the bitstream.
6. Once the authencaon is successful, FSBL starts copying 56 KB starng from the rst block
which is located in DDR/FLASH to ReadBuer, calculates the hash, and then compares it
with the hash stored at HashsOfChunks.
7. If hash comparison is successful, FSBL transmits data to PCAP through DMA (for
unencrypted bitstream) or AES (if encrypon is enabled).
8. FSBL repeats the previous two steps unl the enre 8 MB block is completed.
9. Repeats the enre process for all the blocks of bitstream.
Note: If there is any failure at any stage, PL is reset and FSBL is exited.
The bitstream is directly routed to PCAP through CSU DMA by conguring secure stream switch.
For a DDR system, the whole encrypted bitstream is copied to DDR. For DDR-less system,
decrypon is copied to OCM(TF-A secon) in chunks.
Note: Xilinx recommends that you have a bitstream paron immediately aer the FSBL paron in the
boot image.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 112
Run-Time Security
Run-me security involves protecng the system against incorrectly programmed or malicious
devices corrupng the system memory or causing a system failure.
To protect the system, it is important to secure memory and the peripherals during a soware
execuon. The Zynq UltraScale+ MPSoCs provide memory and peripheral protecon through the
following blocks:
• Trusted Firmware-A
• Xilinx Memory Protecon Unit
• Xilinx Peripheral Protecon Unit
• System Memory Management Unit
• A53 Memory Management Unit
• R5 Memory Protecon Unit
One of the runme security features is access controls on the PMU and CSU global registers
from Linux. These registers are classied into two lists:
• The white list which is accessible all the me by default.
• The black list which is accessible only when a compile me ag is set.
Trusted Firmware-A
The Zynq UltraScale+ MPSoC incorporates the standard execuon model advocated for Armv8
cores. This model runs the normal operang system at a less privileged level, requiring it to
request access to security-sensive hardware or registers using a proxy soware called a secure
monitor. The specic secure monitor provided by Xilinx for the Zynq UltraScale+ MPSoC device
is a part of Linaro Trusted Firmware-A (TF-A). Xilinx neither requires nor provides a Trusted OS
However, the TF-A provided by Xilinx does include hooks that allow customers to add their own
Trusted OS and Trusted applicaons in order to implement a Trusted Execuon Environment. TF-
A is the secure monitor that provides switching between the secure and the non-secure world.
See WP512 for more informaon.
The primary purpose of TF-A is to ensure that the system modules (drivers, applicaons) are
isolated from security resources. For example, Linux should be prevented from accessing the
region where a private key is stored in the SoC. Likewise, the driver for a crypto block does not
need to know the current session key; the session key could be programmed by the key
negoaon algorithm and stored in a secure locaon within the crypto block.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 113
Another usage of TF-A is to prevent any user space applicaon from directly accessing the
hardware cryptographic engine. Instead, a user space applicaon can make a call to the kernel
where the data to be processed is copied to kernel space. Aerwards, the driver will copy the
data from the kernel's virtual memory to physical address. Later, the driver will make a call to TF-
A and then to the PMU/CSU to perform cryptographic operaons on the physical address. For
more informaon, see WP512.
PSCI is the interface from non-secure soware to rmware implemenng power management
use-cases (for example, secondary CPU boot, hotplug, and idle). It might be necessary for
supervisory systems to perform acons, such as restoring context and switches to the power
state of core. Non-secure soware can call TF-A runme services using the Arm secure monitor
call (SMC) instrucon.
In the Arm architecture, synchronous control transfers between the non-secure state to a secure
state are handled through SMC excepons, which are generated by the SMC instrucon. These
are handled by the secure monitor. The operaon of the secure monitor is determined by the
parameters passed in through registers.
Two types of calls are dened:
• Fast calls to execute atomic secure operaons
• Standard calls to start preempve secure operaons
Two calling convenons for the SMC instrucon denes two funcon ideners for the SMC
instrucon dene two calling convenons:
• SMC32: A 32-bit interface that either 32-bit or 64-bit client code can use. SMC32 passes up
to six 32-bit arguments.
• SMC64: A 64-bit interface used only by 64-bit client code that passes up to six 64-bit
arguments.
You dene the SMC funcon ideners based upon the calling convenon. When you dene the
SMC funcon idener, you pass that idener into every SMC call in register R0 or W0, which
determines the following:
• Call type
• Calling convenon
• Secure funcon to invoke
TF-A implements a framework for conguring and managing interrupts generated in either
security state. It implements a subset of the trusted board boot requirements (TBBR) and the
plaorm design document (PDD) for Arm reference plaorms.
The cold boot path is where the TBBR sequence starts when the plaorm is powered on, and
runs up to the stage where it hands-o control to rmware running in the non-secure world in
DRAM. The cold boot path starts when you physically turn on the plaorm.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 114
• You chose one of the CPUs released from reset as the primary CPU, and the remaining CPUs
are considered secondary CPUs.
• The primary CPU is chosen through plaorm-specic means. The cold boot path is mainly
executed by the primary CPU, other than essenal CPU inializaon executed by all CPUs.
• The secondary CPUs are kept in a safe plaorm-specic state unl the primary CPU has
performed enough inializaon to boot them.
For a warm boot, the CPU jumps to a plaorm-specic address in the same processor mode as it
was when released from reset.
TF-A Functions
The following table lists the TF-A funcons:
Note: bl31 is equivalent to the TF-A, which in this case is the secure monitor.
Table 31: TF-A Functions
TF-A Functions Description
bl31_arch_setup(); Generic architectural setup from EL3.
bl31_platform_setup(); Platform setup in BL1.
bl31_lib_init(); Simple function to initialize all BL31 helper libraries.
cm_init(); Context management library initialization routine.
dcsw_op_all(DCCSW); Cleans caches before re-entering the non-secure software
world.
(*bl32_init)(); Function pointer to initialize the BL32 image.
runtime_svc_init(); Calls the initialization routine in the descriptor exported by a
runtime service. After a descriptor is validated, its start and
end owning entity numbers and the call type are combined
to form a unique oen. The unique oen is an index into the
rt_svc_descs_indices array. This index stores the index of the
runtime service descriptor.
validate_rt_svc_desc();
Simple routine to sanity check a runtime service descriptor
before it is used.
get_unique_oen(); Gets a unique oen.
bl31_prepare_next_image_entry(); Programs EL3 registers and performs other setup to enable
entry into the next image after BL31 at the next ERET.
bl31_get_next_image_type(); Returns the next_image_type.
bl31_plat_get_next_image_ep_info (image_type); Returns a reference to the entry_point_info structure
corresponding to the image that runs in the specified
security state.
get_security_state () Gets the security state.
cm_init_context() Initializes a cpu_context for the first use by the current CPU,
and sets the initial entry point state as specified by the
entry_point_info structure.
get_scr_el3_from_routing_model() Returns the cached copy of the SCR_EL3 which contains the
routing model (expressed through the IRQ and FIQ bits) for
a security state that is stored through a previous call to
set_routing_model().
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 115
Table 31: TF-A Functions (cont'd)
TF-A Functions Description
cm_prepare_el3_exit() Prepares the CPU system registers for first entry into the
secure or the non-secure software world.
• If execution is requested to EL2 or hyp mode SCTLR_EL2
is initialized.
• If execution is requested to the non-secure EL1 or svc
mode, and the CPU supports EL2; then EL2 is disabled by
configuring all necessary EL2 registers.
For all entries, the EL1 registers are initialized from the
cpu_context.
cm_get_context(security_state);
Gets the context of the security state.
el1_sysregs_context_restore Restores the context of the system registers.
cm_set_next_context Programs the context used for exception return. This
initializes the SP_EL3 to a pointer to a cpu_context set for the
required security state.
bl31_register_bl32_init Initializes the pointer to BL32 init function.
bl31_set_next_image_type Accessor function to help runtime services determine which
image to execute after BL31.
For more informaon about TF-A, see Trusted Firmware-A documentaon.
FPGA Manager Solution
The FPGA Manager in the Zynq UltraScale+ MPSoC provides an interface to download dierent
types of bitstreams (full, paral, authencated, encrypted and so on) during runme from Linux
environment. The key features of the FPGA Manager are as follows:
• Full bitstream loading
• Paral Reconguraon (paral bitstream loading)
• Encrypted full/paral bitsream loading
• Authencated full/paral bitstream loading
• Authencated and encrypted full/paral bitstream loading
• Readback of conguraon registers
• Readback of bitstream (conguraon data)
FPGA Manager Architecture
The following gure shows the architecture of the FPGA Manager.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 116
Figure 39: FPGA Manager Architecture Block Diagram
Linux FPGA Manager (kernel)
ATF
EEMI/SCMI/SVC
PMUFW
IPI
xilfpga
xilfpga APIs
FPGA/PL
X22151-121818
Execution Flow
FPGA manager provides an abstracon for the user to load bitstream using Linux. The xilfpga
library inializes the PCAP, CSUDMA and other hardware. For more details about xilfpga, see the
XilFPGA secon in the OS and Libraries Document Collecon (UG643).
To load a bitstream, the FPGA manager allocates the required memory and invokes the EEMI API
using the FPGA LOAD API ID. This request is a blocking call. The FPGA Manger waits for
response from the TF-A and response is provided to the fpga core layer which passes it to the
applicaon. This is described in the following gure:
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 117
Figure 40: FPGA Manager Flow
fpgautil -b system.bit
Application Layer
FPGA Manager Core Framework
Linux Kernel — EL1 NS
ZYNQMP FPGA Manager driver
ZYNQMP Firmware driver
ATF
PMUFW
Xilfpga Library
ATF (BL31) — EL3
PMUFW and XILFPGA
Write Request
SMC CALL
IPI
EEMI Request
X22152-110320
Xilinx Memory Protection Unit
The Xilinx memory protecon unit (XMPU) is a region-based memory protecon unit. For more
details, see “System Protecon Unit” chapter in the Zynq UltraScale+ Device Technical Reference
Manual (UG1085).
Protecting Memory with XMPU
Isolaon of memory is fundamental to any secure or funconally safe system. The XMPU gives
users the ability to paron user-dened regions of memory and allocate them to specic
isolated subsystems.
To understand more about XMPU features and funconality, refer to “System Protecon Unit”
chapter in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 118
Configuring XMPU Registers
The XMPU is congurable either one-me or through TrustZone access from a secure master
(PMU, APU TrustZone secure master, or RPU when congured as secure master). At boot me,
the FSBL congures the XMPU and its conguraon can be locked such that it can only be
recongured at next power-on reset. If the conguraon is not locked, then XMPU can be
recongured any number of mes by secure master accesses. If you choose to congure the
XMPU dynamically, you must also consider many aspects including the idling of acve devices
and the AXI bus.
For more informaon on using the XMPU please see Isolaon Methods in Zynq UltraScale+
MPSoCs (XAPP1320).
Xilinx Peripheral Protection Unit
The XPPU allows protecng peripherals, message buers, inter-processor interrupts (IPI),
communicaons, and Quad SPI ash memory. To understand more about Xilinx peripheral
protecon unit (XPPU) features and funconality, see this link to the “Xilinx Peripheral Protecon
Unit” secon of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
For more informaon on using the XMPU please see Isolaon Methods in Zynq UltraScale+
MPSoCs (XAPP1320).
System Memory Management Unit
The system memory management unit (SMMU) provides isolaon to I/O and DMA capable
devices by virtualizing their address space. The SMMU provides address translaon for an I/O
device to idenfy more than its actual addressing capability. In absence of memory isolaon, I/O
devices can corrupt system memory. The SMMU provides device isolaon to prevent DMA
aacks. To oer isolaon and memory protecon, it restricts device access for DMA-capable I/O
to a pre-assigned physical space.
To understand more about SMMU features and funconality, see this link to the “System
Memory Management Unit” secon of the Zynq UltraScale+ Device Technical Reference Manual
(UG1085).
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 119
A53 Memory Management Unit
The memory management unit (MMU) controls table-walk hardware that accesses translaon
tables in main memory. The MMU translates virtual addresses to physical addresses and provides
congurable 1-stage or 2-stage address translaon. The MMU provides ne-grained memory
system control through a set of virtual-to-physical address mappings and memory aributes held
in page tables. These are loaded into the translaon lookaside buer (TLB) when a locaon is
accessed.
To understand more about MMU features and funconality, see this link to the “Memory
Management Unit” secon of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
R5 Memory Protection Unit
The memory protecon unit (MPU) enables you to paron memory into regions and set
individual protecon aributes for each region. When the MPU is disabled, no access permission
checks are performed, and memory aributes are assigned according to the default memory map.
The MPU has a maximum of 16 regions.
To understand more about MPU features and funconality, see this link to the “Memory
Protecon Unit” secon of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
TrustZone
TrustZone provides state isolaon, which can divide peripherals, memory, and applicaons into a
secure and non-secure world. See Isolaon Methods in Zynq UltraScale+ MPSoCs (XAPP1320) for
details and the TRM.
Chapter 8: Security Features
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 120
Chapter 9
Platform Management
Zynq
®
UltraScale+™ MPSoCs are designed for high performance and power-sensive
applicaons in a wide range of markets. The system power consumpon depends on how
intelligently soware manages the various subsystems – turning them on and o only when they
are needed and, also at a ner level, trading o performance for power. This chapter describes
the features available to manage power consumpon, and how to control the various power
modes using soware.
Platform Management in PS
To increase the scalability in the plaorm management unit (PMU), the Zynq UltraScale+ MPSoC
supports mulple power domains such as:
• Full Power Domain
• Low Power Domain
• Baery Power Domain
• PL Power Domain
For details on the PMU and the oponal PMU rmware (PMU rmware) funconality, see the
Zynq UltraScale+ Device Technical Reference Manual (UG1085).
For more informaon on dynamically changing the PS clocks, see Chapter 14: Clock and
Frequency Management.
The PS block oers high levels of funconality and performance. At the same me, there is a
strong need to opmize the power consumpon of this block with respect to the funconality
and performance that is necessary at each stage of the operaon.
The Zynq UltraScale+ MPSoC has mulple power rails. Each rail can be turned o independently,
or can use a dierent voltage. Many of the blocks on a specic power rail implement power-
gang, which allows blocks to be gated o independently.
Examples of these power-gated domains are the: Arm
®
Cortex
®
-A53 and the Cortex
®
-R5F
processors, GPU pixel processors (PP), large RAMs, and individual USBs.
The following gure shows a block diagram of the plaorm management at the PS level.
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 121
Figure 41: Platform Management at the PS Level
SoC Debug
Quadcore APU
L2
RAM
GPU
Interconnect IPI
IOU CSU PMU
eFuse
AMS
BBRAM RTC
ADMA
SLCR
PLLs
Dual
R5
TCM
GIC
RPU
O
C
M
USB
OSC
PS
PL
DAP, RPU Debug
BPU
Low power domain
Battery power domain
Full power domain
X19226-071317
From the power perspecve, Zynq UltraScale+ MPSoCs oers the following modes of operaon
at the PS level:
• Full-power operaon mode
• Low-power operaon mode
• Deep-sleep mode
• Shutdown mode
• Baery-power mode
The following secons describe these modes.
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 122
Full-Power Operation Mode
In the full-power operaon mode (shown as full power domain in the gure above), the enre
system is up and running. Total power dissipaon depends on the number of components that
are running: their states and their frequencies. In this mode, dynamic power will likely dominate
the total power dissipaon.
To opmize stac and dynamic power in full-power mode, all large modules have their own
power islands to allow them to be shut down when they are not being used. To understand about
full-power operaon mode, see this link to the “Plaorm Management Unit” chapter in the Zynq
UltraScale+ Device Technical Reference Manual (UG1085).
Low-Power Operation Mode
In the low-power operaon mode, a subset of the PS (shown as low-power domain in the gure
above) is powered up that includes: the PMU, RPU, CSU, and the IOU.
In this mode, the ability to change system frequency allows power dissipaon to be tuned. The
CSU must be running connuously to monitor the system security against SEU and tampering. In
this mode, the ability to change system frequency allows power dissipaon to be tuned.
The low-power mode includes all lower-domain peripherals. Among the blocks within the low-
power mode, PLLs, dual Cortex-R5F, USBs, and the TCM and OCM block RAMs oer power
gang.
Note: SATA, PCIe
®
, and DisplayPort blocks are within the full power domain (FPD).
You can control power gang to dierent blocks through soware by conguring the LPD_SLCR
registers. See the SLCR_Registers link in the Zynq UltraScale+ Device Register Reference (UG1087)
for more informaon on LPD_SLCR register.
Deep-Sleep Operation Mode
Deep-Sleep is a special mode in which the PS is suspended and waing a wake-up signal. The
wake can be triggered by the MIO, the USB, or the RTC.
Upon wake, the PS does not have to go through the boot process, and the security state of the
system is preserved. The device consumes the lowest power during this mode while sll
maintaining its boot and security state.
In this mode, all the blocks outside the low-power domain, such as the system monitor and PLLs,
are powered down. In LPD, Cortex-R5F is powered down. Because this mode has to preserve the
context, TCM and OCM are in a retenon state.
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 123
Shutdown Mode
Shutdown mode powers down the enre APU core. This mode is applicable to APU only. During
shutdown, the enre processor state, including its caches, is completely lost; therefore, soware
is required to save all states before requesng the PMU to power down the APU core.
When a CPU is shutdown, it is expected that any interrupt from a peripheral that is associated
with that CPU to iniate its power up; therefore, the interrupt lines to an APU core are also
routed to the PMU interrupt controller, and are enabled when the APU core is powered down.
The Embedded Energy Management Interface EEMI API Reference Guide (UG1200) describe the
APIs to invoke shutdown.
For more details, see this link to the “Plaorm Management Unit Programming Model” secon in
the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Battery-Powered Mode
When the system is OFF, limited funconality within the PS must stay ON by operang on a
baery. The following features operate within the baery-powered domain PS:
• Baery-backed RAM (BBRAM) to hold key for secure conguraon
• Real-me clock (RTC) including the crystal I/O
The Zynq UltraScale+ MPSoC includes only one baery-powered domain and only the funcons
those are implemented in the PS can be baery backed-up. The required I/O for the baery-
powered domain includes the baery power pads and the I/O pads for the RTC crystal.
Power Management Framework
The Embedded Energy Management Interface EEMI API Reference Guide (UG1200) describes how to
use the power API funcons.
Note: There is no dierence between bare metal, FreeRTOS, or Linux-specic power management Xilinx
EEMI API oerings.
Wake Up Mechanisms
To understand about wake up mechanisms, see this link to the “Plaorm Management Unit
Operaon” secon of “Chapter 6, Plaorm Management Unit” of theZynq UltraScale+ Device
Technical Reference Manual (UG1085).
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 124
Platform Management for Memory
The Zynq UltraScale+ MPSoCs include large RAMs like L2 cache, OCM, and TCM. These RAMs
support various power management features such as: clock gang, power gang, and memory
retenon modes.
• TCM and OCM support independent power gang and retenon modes.
• The L2 cache controller supports dynamic clock gang, retenon, and shutdown modes to
reduce power consumpon at a ner granularity.
DDR Controller
The DDR controller implements the following mechanisms to reduce its power consumpon:
• Clock Stop: When enabled, the DDR PHY can stop the clocks to the DRAM.
• For DDR2 and DDR3, this feature is only eecve in self-refresh mode.
• For LPDDR2, this feature becomes eecve during idle periods, power-down mode, self-
refresh mode, and deep power-down mode.
• Pre-Charge Power Down: When enabled, the DDRC dynamically uses pre-charge power
down mode to reduce power consumpon during idle periods. Normal operaon connues
when a new request is received by the controller.
• Self-Refresh: The DDR controller can dynamically put the DRAM into self-refresh mode
during idle periods. Normal operaon connues when a new request is received by the
controller.
In this mode, DRAM contents are maintained even when the DDRC core logic is fully powered
down; this allows stopping the DDR3X clock and the DCI clock that controls the DDR
terminaon.
Platform Management for Interconnects
The Interconnect lays across mulple power rails and power islands which can be on or o at
dierent mes. To ease the implementaon, in most cases, the clocks for two power domains
that communicate with one another must be asynchronous; consequently, requiring
synchronizers on their interconnecon.
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 125
To ease ming, the power domain is placed exactly at the clock crossing. The synchronizer must
be implemented as two separate pieces with each placed in one of the two domains that are
connected through the synchronizer, creang a bridge.
The bridge consists of a slave interface and a master interface with each lying enrely within a
single power and clock domain. The clock frequencies at the interfaces can vary independent of
each other, and each half can be reset independent of the other half.
Level shiers or clamping, or both, must be implemented between the two halves of the bridge
for mul-voltage implementaon or power-o.
Also, the bridge keeps track of open transacons, as follows:
• When the bridge receives a power-down request from the PMU, it logs that request.
• All new transacons return an error while the previously open transacons are being
processed as usual unl the transacon counter becomes 0. At that point, the bridge
acknowledges to the PMU that it is safe to shut down the master or slave connected to the
bridge.
• The enre Interconnect shuts down only when all bridges within that interconnect are idle.
For more details, see this link to the “PMU Interconnect” sub-secon in the “Plaorm
Management Unit” chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
PMU Firmware
Every system conguraon that is supported by Xilinx includes PMU rmware in addion to the
funcons of power-up and sleep management. The PMU can execute user programs that
implement advanced system monitoring and power management algorithms. In this mode, an
applicaon or a real-me processor copies the power management program into the PMU
internal RAM through an inbound LPD switch. The PMU executes soware that implements the
required reset, power management, system monitoring, and interrupt controls within all Xilinx
supported system conguraons.
For more details, see this link to the “Plaorm Management Unit Programming Model” secon in
“Chapter 6” of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
You can use the Vis soware plaorm to create custom PMU rmware. It provides the source
code for the PMU rmware template and the necessary library support. For details on how to
create a Vis project, see Chapter 5: Soware Development Flow.
Chapter 9: Platform Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 126
Chapter 10
Platform Management Unit
Firmware
The Plaorm Management Unit (PMU) in Zynq
®
UltraScale+™ MPSoCs is located within the
Low-power sub-system. The PMU consists of a MicroBlaze™ processor which loads executable
code from 32 KB ROM and 128 KB RAM into at memory space. The PMU controls the power-
up, reset, and monitoring of resources within the system including inter-processor interrupts and
power management registers. The ROM is preloaded with PMU bootROM (PBR) which performs
pre-boot tasks and enters a service mode. PMU_FW must be loaded to provide advanced system
funconality for each of the Xilinx
®
supported use-cases. This chapter explains the features and
funconality of PMU rmware developed for Zynq UltraScale+ MPSoC.
Features
The following are the key features of PMU rmware:
• Provides modular funconality: PMU rmware is designed to be modular. It enables you to
add a new funconality in the form of a module
• Provides easy customizaon of modules
• Easily congurable to include only the required funconality for a user applicaon
• Support communicaon with other components in the system over IPI (Inter-Processor
Interrupt)
• Run me congurability for EM module
• Support for various Power Management features
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 127
PMU Firmware Architecture
The following gure shows the architecture block diagram of PMU rmware. PMU rmware is
designed to be modular and enables adding new funconality in the form of modules. Each
funconally disnct feature is designed as a module so that the PMU rmware can be congured
to include only the required funconality for a user applicaon. This type of modular design
allows easy addion of new features and opmizes memory footprint by disabling unused
modules.
PMU rmware can be divided into base rmware and modules. PMU Base Firmware does
inializaon of modules, registering events for the modules, and provides all the common
funcons that may be required by the modules. These common funcons can be categorized into
the following APIs:
1. PMU rmware Core APIs
a. Scheduler
b. Event Manager
c. IPI Manager
2. PMU rmware General APIs
a. BSP/Ulity APIs
b. Reset Services APIs
c. ROM Services APIs
These APIs can be used by the modules in PMU rmware to perform the specied acons as
required.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 128
Figure 42: PMU firmware Architecture Block diagram
Execution Flow
The inializaon in PMU rmware takes place in a normal context. Interrupts are disabled to
avoid unintended interrupons and prevent usage of the system resources before they are
properly inialized. Aer inializaon completes, interrupts are enabled and the required tasks
are scheduled to be executed. The system enters in to a sleep state. The system wakes up only
when an event occurs or the scheduled tasks are triggered and the corresponding handlers are
executed. The following gure shows the state transions for PMU rmware.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 129
Figure 43: State Transitions for PMU firmware in Main Loop
Main Loop
[Active]
Execute Triggered
Tasks
[Active]
Execute Interrupt
Handlers
Wake on Interrupt
Other Wakes
[Sleep]
Main Loop
[Active]
Execute Triggered
Tasks
[Active]
Execute Interrupt
Handlers
[Sleep]
X24857-120120
PMU rmware execuon ow consists of the following three phases:
• Inializaon phase: This phase consists of PMU rmware starng up, performing self-tests
and validaons, inializing the hardware, creang and inializing modules. Interrupts are
disabled during this phase and are enabled at the end.
• Post inializaon: In this phase, PMU rmware enters service mode, wherein it enters into
sleep and waits for an interrupt.
• Waking up: PMU rmware enters the interrupt context and services the interrupt. Aer
compleng this task, it goes back to sleep.
The following gure shows the execuon ow for PMU rmware.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 130
Figure 44: Execution Context View for PMU firmware
Interrupt ContextMain Loop
Start Up
Create Modules
Initialize Modules
Enable Interrupts
Sleep
Execute Triggered
Tasks
Interrupt
Handler
-Dispatch Events to registered
modules
-Modules execute the handlers
-Dispatch IPI to registered
modules
-Modules execute IPI handlers
PIT:
-Flag tasks for execution
interrupt
reti
interrupt
reti
X24856-120120
Handling Inter-Process Interrupts in PMU
firmware
IPI is a key interface between PMU rmware and non-PMU enes on the SoC. PMU includes
four Inter-Processor Interrupts (IPI) assigned to it and one set of buers. PMU rmware uses
IPI-0 and associated buers for communicaon by default, which is iniated by other masters on
SoC to PMU. PMU rmware uses IPI-1 and associated buers for callbacks from PMU to other
masters and for communicaon iniated by PMU rmware.
The following gure shows the IPI handling stack with interfaces between dierent components
involved in this process. PMU rmware uses IPI driver to send and receive the messages. An IPI
manager layer in Base Firmware is implemented over the driver and it takes care of dispatching
the IPI message to the registered module handlers based on IPI ID in the rst word of the
message. The following table displays the message format for IPI.
Table 32:
IPI Message Format
Word Content Description
0 Header <target_module_id, api_id>
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 131
Table 32: IPI Message Format (cont'd)
Word Content Description
1 Payload Module dependent payload
2
3
4
5
6 Reserved Reserved - for future use
7 Checksum
IPI-1 is used for the callbacks from PMU to other masters and for communicaon iniated by
PMU rmware. Currently, PM and EM modules use IPIs and this can be taken as reference for
implemenng custom modules which require IPI messaging.
Figure 45: IPI Handler Stack with Interfaces
Core Framework
IPI Manager
IPI Driver
Module
GetSource
Write Buffer
TriggerIPI
Read Buffer
GetIpiMask
GetMsg
SendMsg
Dispatch IPI
X22155-121818
PMU rmware provides wrapper APIs around IPI driver funcons to send and receive IPI
messages. During inializaon, PMU rmware inializes the IPI driver and enables IPI interrupt
from the masters which are IPI assigned.
Send IPI Message
XPfw_IpiWriteMessage() API is used to send IPI message to target. This funcon internally
calls the IPI driver write API with buer type as Message buer.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 132
Parameters
Table 33: Send IPI Message
Parameter Description
ModPtr Module pointer from where the IPI message is being sent. In IPI
message, target_module_id field will be updated with the Module IPI ID
information which is present in Module pointer.
DestCpuMask Destination target IPI ID
MsgPtr Message Pointer
MsgLen Message Length
Return
XST_SUCCESS: If message is sent successfully.
XST_FAILURE: If message fails.
Send IPI Response
XPfw_IpiWriteResponse() API is used to send the response to the master which sent an IPI
message. This funcon internally calls the IPI driver write API with buer type as Response
buer.
Parameters
Table 34: Send IPI Response
Parameter Description
ModPtr Module pointer to check which module received this IPI response
SrcCpuMask Source IPI ID to read IPI response
MsgPtr Response Message Pointer
MsgLen Response Message Length
Return
XST_SUCCESS: If IPI response is read successfully.
XST_FAILURE: If response fails.
Read IPI Message
XPfw_IpiReadMessage() is used to read the IPI message received when IPI interrupt comes.
This funcon internally calls the IPI driver read API with buer type as Message buer.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 133
Parameters
Table 35: Read IPI Message
Parameter Description
SrcCpuMask Source IPI ID to read the IPI message
MsgPtr Message Pointer
MsgLen Message Length
Return
XST_SUCCESS: If IPI message is read successfully.
XST_FAILURE: If message fails.
Read IPI Response
XPfw_IpiReadResponse() is used to read the IPI response for the message sent. This
funcon internally calls the IPI driver read API with buer type as response buer.
Parameters
Table 36: Read IPI Response
Parameter Description
ModPtr Module pointer to check which module received this IPI response
SrcCpuMask Source IPI ID to read IPI response
MsgPtr Response Message Pointer
MsgLen Response Message Length
Return
XST_SUCCESS: If IPI response is read successfully.
XST_FAILURE: If response fails.
Triggering an IPI
XPfw_IpiTrigger() is used to trigger an IPI to the desnaon. This funcon internally calls
the IPI driver trigger. This funcon should be called aer the IPI message writes IPI buer.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 134
Parameters
Table 37: Triggering an IPI
Parameter Description
DestCpuMask Destination target IPI ID
Return
XST_SUCCESS: If IPI is triggered successfully.
XST_FAILURE: If trigger fails.
Note: Vivado
®
allows you to enable or disable the IPI. To do so, select MPSoC IP → Re-customize IP → 
Switch To Advanced Mode → Advanced Conguraon → Inter Processor Interrupt (IPI) Conguraon → 
IPI-Master Mapping. However, it is not recommended that you disable IPI channels for APU or RPU for the
PMU rmware PM module to work as expected because in the default conguraon, PM assumes that
both APU and RPU IPI channels are enabled.
PMU Firmware Modules
PMU rmware consists of the following modules:
1. Error Management (EM)
2. Power Management (PM)
3. Scheduler
4. Safety Test Library (STL)
PMU rmware has a module data structure (XPfw_Module_t) which contains the informaon
about the module. This data structure is dened for each module when the module is created.
The following table shows its members.
Table 38: Module Data Structure Members
Member Range Additional Information
ModId 0.. 31
CfgInitHandler Init handler function pointer Default to NULL
IpiHandler Handler for IPI manager Default to NULL
EventHandler Handler for registered events of the
module
Default to NULL
IpiId 16-bit IPI ID Unique to each module
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 135
PMU rmware also has a core data structure which contains the list and the details of all
modules. The following table shows its members.
Table 39: Core Data Structure Members
Member Range Additional Information
ModList array 0.. 31 Module list array (of 32 elements) of
Module structure
Scheduler Scheduler structure Scheduler task owned by the module
ModCount 0.. 31
IsReady Core is ready/dead
Mode Safety Diagnostics mode/Normal mode
Base PMU rmware supports a few APIs that are used by these modules. Also, if you want to
create a custom module, these APIs can be used from xpfw_core.h.
Creating a Module
XPfw_CoreCreateMod() API is called during the startup to create a module. PMU rmware
can have maximum of 32 modules. This funcon checks if the module count reached the
maximum count. If not, it lls in the details to core structure ModList and returns this module
data structure to the caller. Otherwise, it returns NULL.
Setting up Handlers for the Module
Each module can be provided with three handlers which are called during the respecve phases
as described below:
Table 40: Module Handlers
Module
Handler
Purpose API for Registering the Handler
Execution
context
Init Called during the init of the core to
configure the module, register for
events or add scheduler tasks. This
can be used to load the
configuration data into the module
if required.
XPfw_CoreSetCfgHandler(const XPfw_Module_t
*ModPtr,
XPfwModCfgInitHandler_tCfgHandler);
StartUp
Event Handler Called when an event occurs
(module should have registered for
this event, preferably during the init
phase
XPfw_CoreSetEventHandler(const
XPfw_Module_t *ModPtr,
XPfwModEventHandler_t EventHandler);
Interrupt
IPI Handler Called when an IPI message with
respective module-id arrives
XPfw_CoreSetIpiHandler(const XPfw_Module_t
*ModPtr, XPfwModIpiHandler_t IpiHandler,
u16 IpiId);
Interrupt
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 136
PMU Firmware Build Flags
In PMU rmware, each module can be enabled/disabled based on your requirement. This is
achieved by using build ags. The following table describes the important build ags in PMU
rmware and its usage. Please see xpfw_config.h le in PMU rmware sources for a complete
list of build ags.
Table 41: PMU Firmware Build Flags
Flag Description Prerequisites
Default
Setting
XPFW_DEBUG_DETAILED Enables detailed debug prints in PMU
firmware.
This feature is supported in 2017.3 release
and above.
Disabled
PM_LOG_LEVEL Enables print based debug functions for
PM module. Possible values are:
• Alerts
• Errors
• Warnings
• Information
Higher numbers include the debug scope
of lower number, i.e. enabling 3
(warnings) also enables 1 (alerts) and 2
(errors).
Disabled
ENABLE_EM Enables Error Management Module. ENABLE_SCHEDULER Disabled
ENABLE_ESCALATION Enables escalation of sub-system restart to
SRST/PS-Only if the first restart attempt
fails.
ENABLE_RECOVERY,
ENABLE_EM,
ENABLE_SCHEDULER
Disabled
ENABLE_RECOVERY Enables WDT based restart of APU sub-
system.
ENABLE_EM, ENABLE_PM,
EMABLE_SCHEDULER
Disabled
ENABLE_PM Enables Power Management Module Enabled
ENABLE_NODE_IDLING Enables idling and reset of nodes before
force shutdown of a sub-system.
Disabled
ENABLE_SCHEDULER Enables Scheduler module Enabled
ENABLE_WDT Enables CSU WDT based restart of system
used by PMU.
ENABLE_SCHEDULER,
ENABLE_EM
Disabled
ENABLE_STL Enables STL Module. None Disabled
ENABLE_RTC_TEST Enables RTC event handler test module. None Disabled
ENABLE_SAFETY Enables CRC calculation for IPI messages. None Disabled
ENABLE_FPGA_LOAD Enables FPGA bit stream loading feature. ENABLE PM Enabled
ENABLE_SECURE Enables security features. ENABLE PM Enabled
IDLE_PERIPHERALS Enables idling peripherals before PS-only
or System reset.
ENABLE PM Disabled
ENABLE_POS Enables Power Off Suspend feature. ENABLE PM Disabled
EFUSE_ACCESS Enables efuse access feature. ENABLE PM Disabled
ENABLE_UNUSED_RPU_
PWR_DWN
Powers down RPU(s) and slaves if they are
not running after receiving PmInitFinalize.
Enabled
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 137
Table 41: PMU Firmware Build Flags (cont'd)
Flag Description Prerequisites
Default
Setting
USE_DDR_FOR_APU_RESTART Enables handling of APU restart gracefully
by storing FSBL to DDR during boot and
restoring it back to OCM before
performing APU restart.
ENABLE_SECURE Enabled
Error Management (EM) Module
Error Management Hardware
Zynq UltraScale+ MPSoC has a dedicated error handler to aggregate and handle fatal errors
across the SoC. See the TRM/Arch Spec for more informaon.
All fatal errors routed to Error Manager can either set to be handled by HW (and trigger a
SRST/PoR/PS error out) or trigger an interrupt to PMU.
Error Management in PMU firmware
Error management module inializes and handles the errors that are generated by hardware and
provides an opon for you to customize these handlers. In hardware, there are two error status
registers which hold the type of error that occurred. Also any error can be enabled/disabled from
interrupng the PMU MicroBlaze. For each of the errors, you can decide what acon should be
taken when the error occurs. The possible scenarios would be one or a combinaon of the
following choices:
1. Asserng of PS_ERROR_OUT signal on the device
2. Generaon of an interrupt to the PMU processor
3. Generaon of a system reset (SRST)
4. Generaon of a power-on-reset (POR)
PMU rmware provides APIs to register custom error handlers or assign a default (SRST/PoR/PS
error out) acon in response to an Error. When PMU rmware starts, it sets an error acon as
interrupt to PMU for some of the errors and PS error out for others as per the ErrorTable[]
structure dened in xpfw_error_manager.c.
Error Management API Calls
This secon describes the APIs supported by Error Management module in PMU rmware.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 138
Setting up Error Action
XPfw_EmSetAction() API is used to setup an acon for the specied error.
Parameters
Table 42: XPfw_EmSetAction
Parameter Description
ErrorId ErrorId is ID for error as defined in EM Error ID Table.
ActionId ActionId is one of the actions defined in EM Error Action
Table.
ErrorHandler ErrorHandler is the handler to be called in case where action
is interrupt to PMU
Return
XST_SUCCESS: If error acon is set properly.
XST_FAILURE: If error acon fails.
Removing Error Action
XPfw_EmDisable() API is used to remove error acon for the specied error.
Parameters
Table 43: XPfw_EmDisable
Parameter Description
ErrorId ErrorId is ID for error to remove error action
Return
XST_SUCCESS: If successful.
XST_FAILURE: If acon fails.
Processing an Error
XPfw_EmProcessError() API processes the errors that occur. If the respecve error is
registered with an error handler, then this funcon will call the respecve handler to take
appropriate acon.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 139
Parameters
Table 44: XPfw_EmProcessError
Parameter Description
ErrorType Type of error received
(EM_ERR_TYPE_1: For errors in PMU GLOBAL
ERROR_STATUS_1
EM_ERR_TYPE_2: For errors in PMU GLOBAL
ERROR_STATUS_2)
Return
XST_SUCCESS: If successful.
XST_FAILURE: If acon fails.
IPI Handling by EM Module
Along with the PM module, error management module also uses IPI-0 channel for message
exchange. APU and RPU 0/1 masters can communicate to this module using IPI. The
target_module_id in IPI message dierenates which module needs to take an acon based
on the message received. The target_module_id for IPI handler registered for EM module is
0xE. Currently, PMU rmware supports only the messages shown in the following table using IPI.
Table 45: IPI Messages Supported by PMU firmware
S.No IPI Message IPI Message ID/API ID
1 Set error action 0x1
2 Remove error action 0x2
3 Send errors occurred 0x3
Set Error Action
When this IPI message is received from any target to PMU rmware, PMU rmware sets the
error acon for the error ID received in the message. If processing of the message is successful, it
sends SUCCESS (0x0) response to the target. Otherwise FAILURE (0x1) response will be sent.
The message format for the same is as below:
Table 46: Message Format for Error Action
Word Description
0 <target_module_id, api_id>
1 Error ID. See EM Error ID Table for the Error IDs supported.
2 Error Action. See EM Error Action Table for the Error Actions supported.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 140
Remove Error Action
When this IPI message is received from any target to PMU rmware, EM module IPI handler will
remove the error acon for the error ID received. And aer processing the message, it will send
SUCCESS/FAILURE response to the target respecvely. The message format for the same is as
below:
Table 47: Message Format for Removing Error Action
Word Description
0 <target_module_id, api_id>
1 Error ID. See EM Error ID Table for the Error IDs supported.
Send Errors Occurred
PMU rmware saves the errors that occur in the system and sends to the target upon request.
The message format is as below:
Table 48: Message Format for Sending Errors Occurred
Word Description
0 <target_module_id, api_id>
The following table shows the response message sent by PMU rmware.
Table 49: Response Message by PMU Firmware
Word Description
0 <target_module_id, Success/Failure>
1 Error_1 (Bit description is as ERROR_STATUS_1 register in PMU Global registers. If a bit is
set to 1, then it means the respective error as described in ERROR_STATUS_1 has
occurred)
2 Error_2 (Bit description is as ERROR_STATUS_2 register in PMU Global registers. If a bit is
set to 1, then it means the respective error as described in ERROR_STATUS_2 has
occurred)
3 PMU RAM Correctable ECC Count
EM Error ID Table
Table 50: EM Error ID Table
Error ID
Error
Number
Error Description
Default Error
Action
EM_ERR_ID_CSU_ROM 1 Errors logged by CSU bootROM (CBR) PS Error Out
EM_ERR_ID_PMU_PB 2 Errors logged by PMU bootROM (PBR)
in the pre-boot stage
PS Error Out
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 141
Table 50: EM Error ID Table (cont'd)
Error ID
Error
Number
Error Description
Default Error
Action
EM_ERR_ID_PMU_SERVICE 3 Errors logged by PBR in service mode PS Error Out
EM_ERR_ID_PMU_FW 4 Errors logged by PMU firmware PS Error Out
EM_ERR_ID_PMU_UC 5 Un-Correctable errors logged by PMU
HW. This includes PMU ROM validation
Error, PMU TMR Error, uncorrectable
PMU RAM ECC Error, and PMU Local
Register Address Error
PS Error Out
EM_ERR_ID_CSU 6 CSU HW related Errors PS Error Out
EM_ERR_ID_PLL_LOCK 7 Errors set when a PLL looses lock (These
need to be enabled only after the PLL
locks-up)
PS Error Out
EM_ERR_ID_PL 8 PL Generic Errors passed to PS PS Error Out
EM_ERR_ID_TO 9 All Time-out Errors [FPS_TO, LPS_TO] PS Error Out
EM_ERR_ID_AUX3 10 Auxiliary Error 3 PS Error Out
EM_ERR_ID_AUX2 11 Auxiliary Error 2 PS Error Out
EM_ERR_ID_AUX1 12 Auxiliary Error 1 PS Error Out
EM_ERR_ID_AUX0 13 Auxiliary Error 0 PS Error Out
EM_ERR_ID_DFT 14 CSU System Watch-Dog Timer Error System Reset
EM_ERR_ID_CLK_MON 15 Clock Monitor Error PS Error Out
EM_ERR_ID_XMPU 16 XPMU Errors [LPS XMPU, FPS XPMU] Interrupt to PMU
EM_ERR_ID_PWR_SUPPLY 17 Supply Detection Failure Errors PS Error Out
EM_ERR_ID_FPD_SWDT 18 FPD System Watch-Dog Timer Error Interrupt to PMU if
ENABLE_RECO
VERY flag is defined
and FSBL runs on APU.
Otherwise, System
Reset
EM_ERR_ID_LPD_SWDT
19 LPD System Watch-Dog Timer Error Interrupt to PMU if
ENABLE_RECO
VERY flag is defined
and FSBL runs on RPU.
Otherwise, System
Reset
EM_ERR_ID_RPU_CCF
20 Asserted if any of the RPU CCF errors
are generated
PS Error Out
EM_ERR_ID_RPU_LS 21 Asserted if any of the RPU CCF errors
are generated
Interrupt to PMU
EM_ERR_ID_FPD_TEMP 22 FPD Temperature Shutdown Alert PS Error Out
EM_ERR_ID_LPD_TEMP 23 LPD Temperature Shutdown Alert PS Error Out
EM_ERR_ID_RPU1 24 RPU1 Error including both Correctable
and Uncorrectable Errors
PS Error Out
EM_ERR_ID_RPU0 25 RPU0 Error including both Correctable
and Uncorrectable Errors
PS Error Out
EM_ERR_ID_OCM_ECC 26 OCM Uncorrectable ECC Error PS Error Out
EM_ERR_ID_DDR_ECC 27 DDR Uncorrectable ECC Error PS Error Out
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 142
EM Error Action Table
Table 51: EM Error Action Table
Error Action
Error Action
Number
Error Action Description
EM_ACTION_POR 1 Trigger a Power-On-Reset
EM_ACTION_SRST 2 Trigger a System Reset
EM_ACTION_CUSTOM 3 Call the custom handler registered as ErrorHandler
parameter
EM_ACTION_PSERR 4 Trigger a PS-Error Out action
PMU Firmware Signals PLL Lock Errors on
PS_ERROR_OUT
When EM module is enabled, it is recommended to enable SCHEDULER also. During FSBL
execuon of psu_init, it is expected to get the PLL lock errors. To avoid these errors during
EM module inializaon, PMU rmware will not enable PLL Lock errors. It waits for psu_init
compleon by FSBL using a scheduler task. Aer FSBL completes execuon of psu_init, PMU
rmware will enable all PLL Lock errors.
In xpfw_error_management.c, you can see the following default behavior of the PMU
rmware for PLL Lock Errors:
[EM_ERR_ID_PLL_LOCK] = { .Type = EM_ERR_TYPE_2, .RegMask =
PMU_GLOBAL_ERROR_STATUS_2_PLL_LOCK_MASK, .Action = EM_ACTION_NONE, .Handler
=
NullHandler},
where, PMU_GLOBAL_ERROR_STATUS_2_PLL_LOCK_MASK is #dened with 0X00001F00
value, which means that all the PLL Lock Errors are enabled. Hence, if the design do not use any
PLL/PLLs that are not locked, this triggers the PS_ERROR_OUT signal. It means that the
PMU_GLOBAL.ERROR_STATUS_2 register (bits [12:8]) signals that one or more PLLs are NOT
locked and that triggers the PS_ERROR_OUT signal.
To analyze further and see if this is really an issue is to fully understand the status of the PLL in
the design. For example, if the design only uses IO_PLL and DDR_PLL and
PMU_GLOBAL.ERROR_STATUS_2 register signals 0x1600 value, it means that the RPU_PLL,
APU_PLL, and Video_PLL Lock errors have occurred. Looking at a few more registers, you can
really understand the status of the PLLs.
PLL_STATUS
• PLL_STATUS (CRL_APB) = FF5E0040: 00000019
• PLL_STATUS (CRF_APB) = FD1A0044: 0000003A
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 143
Table 52: PLL_STATUS
PLL STATUS ERROR_STATUS_2
IOPLL is locked and stable Bit [8] is for IO_PLL = 0
RPLL is stabled and NOT locked (which means bypassed) Bit [9] is for RPU_PLL = 1
APPL is stabled and NOT locked (which means bypassed) Bit [10] is for APU_PLL = 1
DPLL is locked and stable Bit [11] is for DDR_PLL = 0
VPLL is stabled and NOT locked (which means bypassed) Bit [12] is for Video_PLL = 1
Hence, if the design only uses IO_PLL and DDR_PLL, then it is not really an error to have
RPU_PLL, APU_PLL and Video_PLL in NOT locked status.
Xilinx recommends you to customize the PMU_GLOBAL_ERROR_STATUS_2_PLL_LOCK_MASK
to cover only the PLL of interest so that you can have a meaningful PS_ERROR_OUT signal.
Example:
#define PMU_GLOBAL_ERROR_STATUS_2_PLL_LOCK_MASK ((u32)0X00000900U) will only
signal on PS_ERROR_OUT IO_PLL and DDR_PLL errors.
Power Management (PM) Module
Zynq UltraScale+ MPSoC Power Management framework is based on an implementaon of the
Embedded Energy Management Interface (EEMI). This framework allows soware components
running across dierent processing units (PUs) on a chip or device to issue or respond to requests
for power management.
The Power Management module is implemented within the PMU rmware as an event-driven
module. Events processed by the Power Management module are called power management
events. All power management events are triggered via interrupts.
When handling an interrupt the PMU rmware determines whether the associated event shall be
processed by the Power Management module. Accordingly, if the PMU rmware determines that
an event is power management related and if the Power Management module is enabled, the
PMU rmware triggers it to process the event.
For example, all the PS and PL interrupts can be routed to the PMU via the GIC Proxy. When the
applicaon processors (APU or RPU) are temporarily suspended, the PMU handles the GIC Proxy
interrupt and wakes up the applicaon processors to service the original interrupts. The PMU
rmware does not actually service these interrupts, although you are free to customize the PMU
rmware so that these interrupts are serviced by the PMU instead of by the applicaon
processors. For more informaon, see the ‘Interrupts’ chapter of the Zynq UltraScale+ Device
Technical Reference Manual (UG1085).
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 144
When processing a power management event the Power Management controller may exploit the
PMU ROM handlers for parcular operaons regarding the state control of hardware resources.
Warm restart and FPGA conguraon manager are part of Power Management module. PMU
rmware includes XilFPGA and XilSecure libraries to support the funconalies of PL FPGA
conguraon and to access secure features respecvely. See Chapter 11: Power Management
Framework for more informaon.
Note: Since the Power Management module uses base rmware APIs such as IPI manager/event manager,
it is not possible to run standalone power management features without PMU rmware. See PM Examples
wiki page for XilPM based design examples.
Scheduler
A scheduler is required by modules like STL in order to support periodic tasks like register
coverage, scrubbing, etc. PMU rmware also uses scheduler for LPD WDT funconality. This will
be explained in the following secon. PMU MicroBlaze has 4 PITs (0-3) and Scheduler uses PIT1.
The scheduler supports up to 10 tasks. The following table shows the Scheduler’s task list data
structure with members.
Table 53: Scheduler Data Structure Members
Member Values/Range Additional information
Task ID 0 0.. 9 0 - Highest priority
Interval Task interval in Milliseconds
OwnerId 0.. 9 Modules that owns this task
Status Enabled/Disabled
Callback Function pointer Default to NULL
Note: By default, scheduler funconality is disabled. To enable the same, ENABLE_SCHEDULER build ag
needs to be dened.
Safety Test Library
Safety Test Library (STL) is a collecon of soware safety mechanisms complemenng hardware
safety features for the detecon of random hardware (HW) faults. PMU rmware has a
placeholder for STL inializaon during PMU rmware startup. This is enabled when
ENABLE_STL build ag is dened. The soware library and the safety documentaon can be
seen at the Safety Lounge.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 145
CSU/PMU Register Access
The following secon discusses how to Read/Write the CSU and PMU global registers and
provides a list of White and Black registers.
Register Write
$ echo > /sys/firmware/zynqmp/config_reg
Register Read
$ echo > /sys/firmware/zynqmp/config_reg
$ cat /sys/firmware/zynqmp/config_reg
CSU and PMU global registers are categorized into two lists:
• By default, the White list registers can be accessed all the me. The following is a list of white
registers.
○ CSU Module:
- Csu_status
- Csu_mul_boot
- Csu_tamper_trig
- Csu__status
- Jtag_chain_status
- Idcode
- Version
- Csu_rom_digest(0:11)
- Aes_status
- Pcap_status
○ PMU Global Module:
- Global_control
- Global_Gen_Storage0 - 6
- Pers_Glob_Gen_Storage0-6
- Req_Iso_Status
- Req_SwRst_Status
- Csu_Br_Error
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 146
- Safety_Chk
• The Black list registers can accessed when a compile me ag is set.
Every other register in both the CSU Module and the PMU_GLOBAL Module that is not covered
in the above white list will be a black register. RSA and RSA_CORE module registers are black
registers.
The #define opon (SECURE_ACCESS_VAL) provides access to the black list. To access black
list registers, build the PMU rmware with SECURE_ACCESS_VAL ag set.
Timers
Zynq UltraScale+ MPSoCs have two system watchdog mers, one each for full-power domain
(FPD) and low-power domain (LPD). Each of these WDT provides error condion informaon to
the error manager. EM module can be congured to set a specic error acon when FPD or LPD
WDT expires. This secon describes the usage of these watchdog mers and the PMU rmware
funconality when these watchdog mers expire.
FPD WDT
FPD WDT can be used to reset the APU or the FPD. PMU rmware error management module
can congure the error acon to be taken when the FPD WDT error occurs. PMU rmware
implemented a recovery mechanism for FPD WDT error. This mechanism is disabled by default.
The same can be enabled by dening ENABLE_RECOVERY build ag.
The EM module in PMU rmware sets FPD WDT error acon as ‘system reset’ when recovery
mechanism is not enabled. In this case, PMU rmware doesn't inialize and congure the FPD
WDT. It is le for Linux driver to inialize and start the WDT if required. When WDT expires,
system restart happens.
When ENABLE_RECOVERY ag is dened and FSBL runs on APU, PMU rmware sets FPD WDT
error acon as ‘interrupt to PMU’ and registers a handler to be called when this error occurs. In
this case, when PMU rmware comes up, it inializes and starts the WDT. It also inializes and
sets the mer mode of TTC to interval mode.
PMU rmware congures FPD WDT expiry me to 60 seconds. And if WDT error occurs, PMU
rmware gets an interrupt and it calls the registered handler. PMU rmware has a restart tracker
structure to track the restart phase and other informaon for a master. APU and RPU are the
masters currently using this structure. Following are its members:
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 147
Table 54: Restart Tracker Structure Members
Member Description
Master Master whose restart cycle is to be tracked
RestartState Track different phases in restart cycle
RestartScope Restart scope upon FPD WDT error interrupt
WdtBaseAddress Base address for WDT assigned to this master
WdtTimeout Timeout value for WDT
ErrorId Error Id corresponding to the WDT
WdtPtr Pointer to WDT for this master
WdtResetId Wdt reset ID
TtcDeviceId TTC timer device ID
TtcPtr Pointer to TTC for this master
TtcTimeout Timeout to notify master for event
TtcResetId Reset line ID for TTC
When WDT error occurs, WDT error handler is called and PMU rmware performs the following:
1. It checks if master is APU and error ID is FPD WDT. Then, it checks if restart state is in
progress or not. If restart state is not in progress, then it changes the restart state to in
progress.
2. Later, it restarts the WDT so that the PMU rmware knows when the WDT error is not due
to APU applicaon.
3. Then, it idles APU by sending an IPI to TF-A through mer interrupt TTC3_0.
Note: This is only true for Linux, and not for bare metal where there is no TF-A.
4. If the rst restart aempt fails, then PMU rmware escalates restart to either system-reset or
PS-only reset if ENABLE_ESCALATION ag is dened. If ENABLE_ESCALATION is not
dened, PMU rmware restarts the APU. Otherwise, PMU rmware performs the following:
• First, PMU rmware checks if PL is congured or not.
• If PL is congured, PMU rmware iniates PS-only restart. Otherwise, it iniates system-
reset.
Note: Ensure that the WDT heartbeat applicaon is running in Linux.
LPD WDT
LPD WDT can be used to reset the RPU. PMU rmware error management module can congure
the error acon to be taken when the LPD WDT error occurs. PMU rmware implements a
recovery mechanism for LPD WDT error. This mechanism is disabled by default. The same can be
enabled by dening the ENABLE_RECOVERY build ag.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 148
The EM module in the PMU rmware sets LPD WDT error acon as "system reset' when
recovery mechanism is not enabled. In this case, PMU rmware doesn't inialize and congure
the LPD WDT. It is le to the RPU user applicaon to inialize and start the WDT, if required.
When WDT expires, the system restarts.
When ENABLE_RECOVERY ag is dened and FSBL is running on RPU, PMU rmware sets FPD
WDT error acon as "interrupt to PMU" and registers a handler to be called when this error
occurs. In this case, when PMU rmware comes up, it inializes and starts the WDT.
PMU rmware congures LPD WDT expiry me to 60s. And if WDT error occurs, PMU rmware
gets an interrupt and it calls the registered handler. PMU rmware maintains a restart tracker
structure for LPD WDT. Refer to Table 10-23 for more informaon.
When WDT error occurs, the WDT error handler is called and PMU rmware performs the
following acons:
1. It checks if master is RPU and error ID is LPD WDT. Then, it checks if restart state is in
progress or not. If restart state is not in progress, then it changes the restart state to in
progress and restarts the WDT to track the next WDT expiry.
2. It applies AIB isolaon for RPU and removes it.
3. If restart scope is set as a subsystem, then it will restart RPU subsystem.
4. If restart scope is set as PS only restart, then PMU rmware will restart PS subsystem.
5. If restart scope is set as system, then it will perform the system restart.
CSU WDT
The CSU WDT is congured to be used by PMU rmware that if PMU rmware applicaon
hangs for some reason, then the system would restart. This funconality is enabled only when
ENABLE_WDT ag is dened.
EM modules sets CSU WDT error acon as ‘System Reset’ Inializaon of CSU WDT depends on
bringing WDT out of reset which is performed by psu_init from FSBL. FSBL writes the status
of psu_init compleon to PMU Global general storage register 5, so that PMU rmware can
check for its compleon before inializing CSU WDT. When ENABLE_WDT ag is dened during
PMU rmware inializaon, it adds a task to scheduler to be triggered for every 100 milli-
seconds unl psu_init compleon status is updated by FSBL. Aer psu_init is completed,
this task will be removed from scheduler tasks list and PMU rmware inializes CSU WDT and
congures it to 90 milli-seconds. It also starts a scheduler task to restart the WDT for every 50
milli-seconds. Whenever CSU WDT error occurs due to PMU rmware code hanging, this error is
handled in hardware to trigger ‘System Reset’ and the system will restart.
Following are the dependencies to use this WDT funconality:
1. EM module needs to be enabled by dening ENABLE_EM ag.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 149
2. ENABLE_WDT ag needs to be dened to use CSU WDT.
3. Scheduler module needs to be enabled by dening ENABLE_SCHEDULER to add a task to
scheduler to check for psu_init compleon and to restart WDT periodically.
Configuration Object
The conguraon object is a binary data object used to allow updang data structures in the
PMU rmware power management module at boot me. The conguraon object must be
copied into memory by a processing unit on the Zynq UltraScale+ MPSoC. The memory region
containing the conguraon object must be accessible by the PMU.
The PMU is triggered to load the conguraon object via the following API call:
XPm_SetConfiguration(address);
The address argument represents the start address of the memory where the conguraon
object is located. The PMU determines the size of the conguraon object based on its content.
Once the PMU loads the conguraon object it updates its data structures which are used to
manage the states of hardware resources (nodes). Paral conguraons are not possible. If the
conguraon object does not provide informaon as dened in this document or provides paral
informaon, the consistency of PMU rmware power management data cannot be guaranteed.
The creator of the conguraon object must ensure the consistency of the informaon provided
in the conguraon object. The PMU does not change the state of nodes once the conguraon
object is loaded. The PMU also does not check whether the informaon about current states of
nodes provided in the conguraon object really matches the current state of the hardware.
Current state is a state of a hardware resource at the moment of processing the conguraon
object by the PMU.
The conguraon object species the following:
• List of masters available in the system
• All the slave nodes the master is currently using and current requirement of the master for the
slave conguraon
• All the slave nodes the master is allowed to use and default requirement of the master for the
slave conguraon
• For each power node, which masters are allowed to request/release/power down
• For each reset line, which masters are allowed to request the change of a reset line value
• Which shutdown mode the master is allowed to request and shutdown meout value for the
master
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 150
• Which masters are allowed to set conguraon aer the conguraon is already set upon the
system boot by the FSBL
PM Configuration Object Generation
PM Conguraon Object is generated as follows:
1. Specify the custom PM framework Conguraon using the PCW tool
2. PCW generates the HDF le
3. At build me, the XSA Parser parses the XSA le and insert the conguraon object into the
FSBL code
Figure 46: Configuration Object Generation
PCW HDF Parser FSBL
HDF Config Object
Initial Configuration at Boot
The conguraon object shall be loaded prior to calling any EEMI API, except the following APIs:
• Get API version
• Set conguraon
• Get Chip ID
Unl the rst conguraon object is loaded the PM controller is congured to inially expect the
EEMI API calls from the APU or RPU master, via IPI_APU or IPI_RPU_0 IPI channels, respecvely.
In other words, the rst conguraon object has to be loaded by APU or RPU.
Aer the rst conguraon object is loaded, the next loading of the conguraon object can be
triggered by a privileged master. Privileged masters are dened in the conguraon object that
was loaded the last.
Following are the steps at boot level:
1. FSBL sends the conguraon object to PMU with the Set Conguraon API
2. PMU parses the conguraon object and congures
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 151
3. PMU powers o all the nodes which are unused aer all the masters have completed the
inializaon
All other requests prior to the rst Set Conguraon API call will be rejected by PMU rmware.
Figure 47: Initial Configuration at Boot
FSBL PMU
PM Init
Set Configuration (ConfigObj)
PMU Firmware Loading Options
PMU rmware can be loaded by either FSBL or CSU BootROM (CBR). Both these ows are
supported by Xilinx. Loading PMU rmware using FSBL has the following benets:
• Possible quick boot me, when PMU rmware is loaded aer bitstream.
• In use cases where you want two BIN les - stable and upgradable, PMU rmware can be part
of the upgradable (by FSBL) image.
IMPORTANT!
CBR loads FSBL. If CBR also loads PMU rmware, it means that the secure headers for
both FSBL and PMU rmware are decrypted with same Key-IV pair, which is a security vulnerability
(security rule is: no two parons should use the same Key-IV pair). This is addressed in FSBL, not in CBR.
Hence, you should avoid CBR loading PMU rmware in secure (decrypon) cases.
For DDR self-refresh over Warm restart, FSBL and PMU rmware must be loaded rst (in any order) before
all other images (e.g. bitstream).
For Power O Suspend, PMU rmware must be loaded rst (i.e. by CSU) before FSBL.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 152
Loading PMU Firmware in JTAG Boot Mode
PM operaons depend on the conguraon object loaded by FSBL from 2017.1 release onwards.
Hence, In JTAG boot mode, it is mandatory to load PMU FW before loading FSBL. In device boot
modes, loading of conguraon object to PMU rmware by FSBL is handled both in CBR loading
PMU rmware and FSBL loading PMU rmware opons. Use the following steps to boot in JTAG
mode:
1. Disable security gates to view PMU MicroBlaze. PMU MicroBlaze is not visible in xsdb for
Silicon v3.0 and above.
2. Load PMU rmware and run.
3. Load FSBL and run.
4. Connue with U-Boot/Linux/user specic applicaon.
Following is a complete Tcl script:
#Disable Security gates to view PMU MB target
targets -set -filter {name =~ "PSU"}
#By default, JTAGsecurity gates are enabled
#This disables security gates for DAP, PLTAP and PMU.
mwr 0xffca0038 0x1ff
after 500
#Load and run PMU FW
targets -set -filter {name =~ "MicroBlaze PMU"}
dow xpfw.elf
con
after 500
#Reset A53, load and run FSBL
targets -set -filter {name =~ "Cortex-A53 #0"}
rst -processor
dow fsbl_a53.elf
con
#Give FSBL time to run
after 5000
stop
#Other SW...
dow u-boot.elf
dow bl31.elf
con
#Loading bitstream to PL
Targets -set -nocase -filter {name =~ "*PL*"}
fpga download.bit
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 153
Loading PMU Firmware in NON-JTAG Boot Mode
When PMU rmware is loaded in a non-JTAG Boot mode on a 1.0 Silicon, an error message
‘Error: Unhandled IPI received’ may be logged by PMU rmware at startup, which can be safely
ignored. This is due to the IPI0 ISR not being cleared by PMU ROM. This is xed in 2.0 and later
versions of silicon.
Using FSBL to Load PMU Firmware
1. Build PMU rmware applicaon in the Vis IDE.
2. Build an FSBL in the Vis IDE for A53. (R5F can also be used).
3. Create a hello_world example for A53.
4. Select Xilinx → Create Boot Image.
5. Create a new bif le. Choose:
a. Architecture: ZynqMP
b. You will see A53 fsbl and hello_world example by default in parons. Also, you need
PMU rmware.
c. Click on Add, then provide pmufw.elf path. Also select Paron type as datale,
Desnaon device as PS, and Desnaon CPU as PMU.
d. Click OK.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 154
6. Aer adding pmufw as paron. Click on pmufw paron and then, click UP.
7. Make sure to select the following paron order:
a. A53 FSBL
b. PMU rmware
c. Hello World applicaon
8. Click on Create Image. You will see BOOT.bin created in a new bootimage folder in your
example project.
9. View the .BIF le to conrm the paron order.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 155
10. Now copy this BOOT.bin into SD card.
11. Boot the ZCU102 board in SD boot mode. You can see the fsbl → pmufw → hello_world
example prints in a sequence.
Using CBR to load PMU Firmware
When PMU rmware is loaded by CBR, it is executed prior to FSBL. So the MIOs, Clocks and
other inializaons are not done at this point. Consequently, the PMU rmware banner and
other prints may not be seen prior to FSBL. Post FSBL execuon, the PMU rmware prints can
be seen as usual.
To make the CBR load PMU rmware, follow these steps:
1. Change the BOOT.bin boot parons.
2. Perform the steps listed in Loading PMU Firmware in NON-JTAG Boot Mode.
3. Create a new bif le. Choose the following:
a. Architecture: ZynqMP.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 156
b. You will see A53 fsbl and hello_world example by default in parons. Also, you need
pmufw.
c. Click Add and then provide the pmufw.elf path. Select the Paron type as pmu
(loaded by bootrom).
d. Click OK.
e. Click on Create Image. You will see BOOT.bin created in a new folder named boomage
in your example project.
f. You can also view .BIF to conrm the paron order.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 157
g. Now copy this BOOT.bin into SD card.
h. Boot the ZCU102 board in SD boot mode. You can see the pmufw → fsbl → hello_world
example prints in a sequence.
PMU Firmware Usage
This secon describes the usage of PMU rmware with examples.
Enable/Disable Modules
This secon describes how to enable/disable PMU rmware build ags both in the Vis soware
plaorm and PetaLinux.
In PetaLinux
1. Create a PetaLinux project.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 158
2. Open <plnx-project-root>/project-spec/meta-user/recipes-bsp/pmu/pmu-
f irmware_%.bbappend le and add the following line:
YAML_COMPILER_FLAGS_append = -DENABLE_EM
The above line enables EM module. To enable any ag, it should be prexed with '-D'.
3. Aer any change to the YAML compiler ags, force a clean state before rebuilding the
applicaon.
Custom Module Usage
Each set of user dened rounes performing a specic funconality should be designed to be a
module in PMU rmware. These les must be self-contained. However, these les can use
declaraons from xpfw_core.h. Each module can register with the following interfaces. If any
of the handler is not needed by the module, it can be skipped from being registered.
• Cong Handler: Called during inializaon.
• Event Handler: Called when a registered event is triggered.
• IPI Handler: Called when an IPI message arrives with the registered IPI ID
Creating a Custom Module
To create a custom module, add the following code to PMU rmware:
/* IPI Handler */
static void CustomIpiHandler(const XPfw_Module_t *ModPtr, u32 IpiNum, u32
SrcMask,
const u32* Payload, u8 Len)
{
/**
* Code to handle the IPI message received
*/
}
/* CfgInit Handler */
static void CustomCfgInit(const XPfw_Module_t *ModPtr, const u32 *CfgData,
u32 Len)
{
/**
* Code to configure the module, register for events or add scheduler tasks
*/
}
/* Event Handler */
static void CustomEventHandler(const XPfw_Module_t *ModPtr, u32 EventId)
{
/**
* Code to handle the events received
*/
}
/*
* Create a Mod and assign the Handlers. We will call this function
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 159
* from XPfw_UserStartup()
*/
void ModCustomInit(void)
{
const XPfw_Module_t *CustomModPtr = XPfw_CoreCreateMod();
(void) XPfw_CoreSetCfgHandler(CustomModPtr, CustomCfgInit);
(void) XPfw_CoreSetEventHandler(CustomModPtr, CustomEventHandler);
(void) XPfw_CoreSetIpiHandler(CustomModPtr, CustomIpiHandler, (u16)IPI_ID);
}
Registering for an Event
All interrupts that come into PMU are exposed to user as Events with specic EVENTIDs dened
in xpfw_events.h. Any module can register for an event (usually in CfgHandler) and the module's
EventHandler will be called when an event is triggered.
To register for an RTC Event:
Status = XPfw_CoreRegisterEvent(ModPtr,XPFW_EV_RTC_SECONDS);
Example of an EventHandler:
void RtcEventHandler(const XPfw_Module_t *ModPtr, u32 EventId)
{
xil_printf("MOD%d:EVENTID: %d\r\n", ModPtr->ModId, EventId);
if(XPFW_EV_RTC_SECONDS == EventId){
/* Ack the Int in RTC Module */
Xil_Out32(RTC_RTC_INT_STATUS,1U);
xil_printf("RTC: %d \r\n", Xil_In32(RTC_CURRENT_TIME));
}
}
Error Management Usage
This secons describes the usage of the EM module to congure the error acon to be taken for
the errors that comes to PMU rmware (the errors generated in the system which are mapped to
PMU MB).
Example for Error Management (Custom Handler)
For this example, OCM uncorrectable error (EM_ERR_ID_OCM_ECC) is considered. The default
error acon for this error is set to PS Error Out. In the following example, a custom handler is
registered for this error in PMU rmware and the handler in this case just prints out the error
message. In a more realisc case, the corrupted memory may be reloaded, but this example is
just limited to clearing the error and prinng a message.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 160
Adding the Error Handler for OCM Uncorrectable ECC in PMU rmware:
+++ b/lib/sw_apps/zynqmp_pmufw/src/xpfw_mod_em.c
@@ -140,6 +140,14 @@ void FpdSwdtHandler(u8 ErrorId)
XPfw_RecoveryHandler(ErrorId);
}
+/* OCM Uncorrectable Error Handler */
+static void OcmErrHandler(u8 ErrorId)
+{
+ XPfw_Printf(DEBUG_DETAILED, "EM: OCM ECC error detected\n");
+ /* Clear the Error Status in OCM registers */
+ XPfw_Write32(0xFF960004, 0x80);
+}
+ /* CfgInit Handler */
static void EmCfgInit(const XPfw_Module_t *ModPtr, const u32 *CfgData,
u32 Len)
@@ -162,6 +170,8 @@ static void EmCfgInit(const XPfw_Module_t *ModPtr,
const u32
*CfgData,
}
}
+ XPfw_EmSetAction(EM_ERR_ID_OCM_ECC, EM_ACTION_CUSTOM, OcmErrHandler);
+
if (XPfw_RecoveryInit() == XST_SUCCESS) {
/* This is to enable FPD WDT and enable recovery mechanism when
To inject OCM Uncorrectable ECC error using debugger (xsdb):
;# Enable ECC UE interrupt in OCM_IEN
mwr -force 0xFF96000C [expr 1<<7]
;# Write to Fault Injection Data 0 Register OCM_FI_D0
;# Errors will be injected in the bits which are set, here its bit0, bit1
mwr -force 0xFF96004C 3
;# Enable ECC and Fault Injection
mwr -force 0xFF960014 1
;
# Clear the Count Register : OCM_FI_CNTR
mwr -force 0xFF960074 0
;# Now write data to OCM for the fault to be injected
# Since OCM does a RMW for 32-bit transactions, it should trigger error here
mwr -force 0xFFFE0000 0x1234
;# Read back to trigger error again
mrd -force 0xFFFE0000
Example for Error Management (PoR as a Response to Error)
Some error may be too fatal and the system recovery from those errors may not be feasible
without doing a Reset of enre system. In such cases PoR or SRST can be used as acons. In this
example use PoR reset as a response to the OCM ECC double-bit error.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 161
Here is the code that adds the PoR as acon:
@@ -162,6 +162,8 @@ static void EmCfgInit(const XPfw_Module_t *ModPtr,
const u32
*CfgData,
}
}
+ XPfw_EmSetAction(EM_ERR_ID_OCM_ECC, EM_ACTION_POR, NULL);
+
if (XPfw_RecoveryInit() == XST_SUCCESS) {
/* This is to enable FPD WDT and enable recovery mechanism when
The Tcl script to inject OCM ECC error is same as the one for above example. Once you trigger
the error, a PoR occurs and you may see that all processors are in reset state similar to how they
would be in a fresh power-on state. PMU RAM also gets cleared o during a PoR. Hence, PMU
rmware needs to be reloaded.
Example for Error Management (PS Error out as a Response to
Error)
If you need to communicate outside of system when any error occurs, PS ERROR OUT response
can be set for that respecve error. So, when that error occurs, error will be propagated outside
and PS_ERROUT signal LED will glow. In this example use PS ERROR OUT as a response to the
OCM ECC double-bit error.
Following is the code that adds the PS ERROR OUT as acon:
@@ -162,6 +162,8 @@ static void EmCfgInit(const XPfw_Module_t *ModPtr,
const u32
*CfgData,
}
}
+ XPfw_EmSetAction(EM_ERR_ID_OCM_ECC, EM_ACTION_PSERR, NULL);
+
if (XPfw_RecoveryInit() == XST_SUCCESS) {
/* This is to enable FPD WDT and enable recovery mechanism when
The Tcl script to inject OCM ECC error is same as the one for above example. Once you trigger
the error, a PS_ERROUT LED will glow on board.
IPI Messaging Usage
This secon describes the usage of IPI messaging from PMU rmware to RPU0 and RPU0 to
PMU rmware. PMU rmware, while inializing IPI driver, also enables IPI interrupt from the IPI
channel assigned master.
From PMU Firmware to RPU0
See Zynq UltraScale Plus MPSoC - IPI Messaging Example for more informaon.
Note: You need to enable EM module in PMU rmware to run this example.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 162
From RPU0 to PMU Firmware
See Zynq UltraScale Plus MPSoC - IPI Messaging Example for IPI messaging example from RPU
to PMU.
IMPORTANT! Since the example in the wiki page shows how to trigger IPI from PMU to RPU0 and vice
versa, to trigger an IPI to/from APU or RPU1, you need to change the desnaon CPU mask to the
intended master.
Adding a Task to Scheduler
Tasks are funcons which take void arguments and return void. Currently PMU rmware has no
way to check that the task returns in a pre-determined me, so this needs to be ensured by the
task design. Consider a task which prints out a message:
void TaskPrintMsg(void)
{
xil_printf("Task has been triggered\r\n");
}
If you want to schedule the above task to occur every 500 ms, the following code can be used.
The TaskModPtr is a pointer for module which is scheduling the task.
Status = XPfw_CoreScheduleTask(TaskModPtr, 500U, TaskPrintMsg);
if(XST_SUCCESS == Status) {
xil_printf("Task has been added successfully !\r\n");
}
else {
xil_printf(Ërror: Failed to add Task !\r\n");
}
Reading FPD Locked Status from RPU
Register 0xFFD600F0 is a local register to PMU rmware, in which bit 31 displays whether FPD
is locked or not locked. (If bit 31 is set to 1, then FPD is locked. It remains isolated unl POR is
asserted). You can verify the FPD locked status by reading this register through PMU rmware.
This can be achieved by an MMIO read call to PMU rmware. Use the following steps to read
FPD locked status from R5:
1. Create an empty applicaon for R5 processor. Enable xilpm library in BSP sengs.
2. Create a new.c le in the project and add the following code:
#include "xipipsu.h"
#include "pm_api_sys.h"
#define IPI_DEVICE_IDXPAR_XIPIPSU_0_DEVICE_ID
#define IPI_PMU_PM_INT_MASKXPAR_XIPIPS_TARGET_PSU_PMU_0_CH0_MASK
#define MMIO_READ_API_ID20U
#define FPD_LOCK_STATUS_REG0xFFD600F0
int main(void)
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 163
{
XIpiPsu IpiInstance; XIpiPsu_Config *Config; s32 Status;
u32 Value;
/* Initialize IPI peripheral */
Config = XIpiPsu_LookupConfig(IPI_DEVICE_ID); if (Config == NULL) {
xil_printf("Config Null\r\n"); goto END;
}
Status = XIpiPsu_CfgInitialize(&IpiInstance, Config, Config-
>BaseAddress);
if (0x0U != Status) { xil_printf("Config init failed\r\n"); goto END;
}
/* Initialize the XilPM library */ Status = XPm_InitXilpm(&IpiInstance);
if (0x0U != Status) {
xil_printf("XilPM init failed\r\n"); goto END;
}
/* Read using XPm_MmioRead() */
Status = XPm_MmioRead(FPD_LOCK_STATUS_REG, &Value); if (0x0U != Status)
{
xil_printf("XilPM MMIO Read failed\r\n"); goto END;
}
xil_printf("Value read from 0x%x: 0x%x\r\n",FPD_LOCK_STATUS_REG, Value);
END:
xil_printf("Exit from main\r\n");
}
Note: This applicaon must be run aer FSBL is successfully executed. This applicaon cannot run
successfully, if FSBL fails to send conguraon object to PMU rmware.
PMU Firmware Memory Layout and Footprint
This secon contains the approximate details of PMU rmware Memory Layout and also the
Memory Footprint with various modules enabled.
In PMU RAM, some part is reserved for PBR leaving around 125.7 KB for PMU rmware. The
following gure shows the memory layout of PMU RAM.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 164
Figure 48: PMU Firmware Memory Layout
128K PMU RAM
32'hFFDC_0000
xxx
xxx
xxx
Reserved: Never Cleared
Stack
xxx
ROM Extension table
_XpbrServExtTbl
Reserved for Firmware
MicroBlaze Reserved
__fw_hw_exception_vector
__fw_interrupt_vector
__fw_sw_exception_vector
__fw_start_vector
0xffdc_0000
fw_start: 0xffdc_0050
0x00
0x08
0x10
0x20
128656 Bytes
1024 Bytes
32 Bytes
244 Bytes
0xffdd_f6e0
0xffdd_fae0
0xffdd_fb00
0xffdd_ff00
0xffdd_fff4
0xffdd_fff8
0xffdd_fffc
0xffdd_0000
X22156-092820
In PMU rmware, only PM module is enabled by default along with Base Firmware and all the
other modules are disabled. See the PMU Firmware Build Flags secon to know about the
default seng of a module.
Note: All the metrics are with compilaon opmized for size -Os. This opmizaon seng is enabled by
default in the Vis IDE. To disable the same, follow the steps menoned in Enable/Disable Modules
secon.
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 165
Table 55: PMU Firmware Metrics
S.No Feature/Component
Size
Occupied
(KB)
Free
Space
(KB)
Additional Notes Remarks
1 PMU firmware without
detailed debug prints
enabled
111.4 16.6 This is with base PMU firmware and
PM module.
2 PMU firmware with detailed
debug prints enabled
115.4 12.6 Detailed debug prints are enabled
when XPFW_DEBUG_DETAILED flag is
defined.
This estimation
is with
combination of
(1) and (2)
3 PMU firmware with Error
Management Module
enabled
114.4 13.6 Error Management module is enabled
when ENABLE_EM and
ENABLE_SCHEDULER flags are defined.
This estimation
is with
combination of
(1) and (3)
4 PMU firmware with Restart
functionality enabled
117.1 10.9 Restart functionality is enabled when
ENABLE_RECOVERY,
ENABLE_ESCALATION and
CHECK_HEALTHY_BOOT flags are
defined along with EMABLE_EM and
ENABLE_SCHEDULER flags.
This estimation
is with
combination of
(1) and (4)
Dependencies
RECOMMENDED:
It is recommended to have all the soware components (FSBL, PMU rmware, TF-A,
U-Boot and Linux) of the same release tag (e.g.: 2017.3).
Chapter 10: Platform Management Unit Firmware
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 166
Chapter 11
Power Management Framework
Introduction
The Zynq
®
UltraScale+™ MPSoC is the industry's rst heterogeneous mulprocessor SoC
(MPSoC) that combines mulple user programmable processors, FPGA, and advanced power
management capabilies.
Modern power ecient designs requires usage of complex system architectures with several
hardware opons to reduce power consumpon as well as usage of a specialized CPU to handle
all power management requests coming from mulple masters to power on, power o resources
and handle power state transions. The challenge is to provide an intelligent soware framework
that complies to industry standard (IEEEP2415) and is able to handle all requests coming from
mulple CPUs running dierent operang systems.
Xilinx has created the Power Management Framework (PMF) to support a exible power
management control through the plaorm management unit (PMU).
This Power Management Framework handles several use case scenarios. For example, Linux
provides basic power management capabilies such as idle, hotplug, suspend, resume, and
wakeup. The kernel relies on the underlining APIs to execute power management decisions, but
most RTOSes do not have this capability. Therefore they rely on user implementaon, which is
made easier with use of the Power Management Framework.
Industrial applicaons such as embedded vision, Advanced Driver Assistance, surveillance,
portable medical, and Internet of Things (IoT) are ramping up their demand for
high-performance heterogeneous SoCs, but they have a ght power budget. Some of the
applicaons are baery operated, and baery life is a concern. Some others such as cloud and
data center have demanding cooling and energy cost, not including their need to reduce
environmental cost. All of these applicaons benet from a exible power management soluon.
Key Features
The following are the key features of the Power Management Framework.
• Provides centralized power state informaon through use of a Plaorm Management Unit
(PMU)
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 167
• Supports Embedded Energy Management Interface (EEMI) APIs (IEEE P2415)
• Manages power state of all devices
• Provides support for Linux power management, including:
○ Linux device tree power management
○ TF-A/PSCI power management support
○ Idle
○ Hotplug
○ Suspend
○ Resume
○ Wakeup process management
• Provides direct control of the following power management features with more than 24 APIs:
○ Processor unit suspend and wake up management
○ Memories and peripherals management
Power Management Software Architecture
The Zynq UltraScale+ MPSoC architecture employs a dedicated programmable unit (PMU) that
controls the power-up, power-down, monitor, and wakeup mechanisms of all system resources.
The customer benets from a system that is beer equipped on handling power management
administraon for a mulprocessor heterogeneous system. However, it is inherently more
complex. The goal of the Power Management Framework is to abstract this complexity, exposing
only the APIs you need to be aware of to meet your power budget goal.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 168
Figure 49: Power Management Framework
Power Management Framework (PMF)
RPU User Application
RTOS & Bare Metal
XilPM APIs
EEMI APIs
PMU Firmware Bare-Metal
PM firmware
EEMI APIs
APU User Application
Linux
ATF/PSCI
Bare Metal & OSes
XilPM APIs
RPU
PMU APU
EEMI APIs
PM
request
PM state
PM
request
PM state
X19504-100620
The intenon of the EEMI is to provide a common API that allows all soware components to
power manage cores and peripherals. At a high level, EEMI allows you to specify a high-level
power management goal such as suspending a complex processor cluster or just a single core.
The underlying implementaon is then free to autonomously implement an opmal power-saving
approach.
The Linux device tree provides a common descripon format for each device and its power
characteriscs. Linux also provides basic power management capabilies such as idle, hotplug,
suspend, resume, and wakeup. The kernel relies on the underlining APIs to execute power
management decisions.
You can also create your own power management applicaons using the XilPM library, which
provides access to more than 24 APIs.
Zynq UltraScale+ MPSoC Power Management
Overview
The Zynq UltraScale+ MPSoC power management framework is a set of power management
opons, based upon an implementaon of the Embedded Energy Management Interface (EEMI).
The power management framework allows soware components running across dierent
processing units (PUs) on a chip or device to issue or respond to requests for power
management.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 169
Zynq UltraScale+ MPSoC Power Management
Hardware Architecture
The Zynq UltraScale+ MPSoC is divided into four major power domains:
• Full power domain (FPD): Contains the Arm
®
Cortex
®
-A53 applicaon processor unit (APU) as
well as a number of peripherals typically used by the APU.
• Low power domain (LPD): Contains the Arm Cortex
®
-R5F real-me processor unit (RPU), the
plaorm management unit (PMU), and the conguraon security unit (CSU), as well as the
remaining on-chip peripherals.
• Programmable logic (PL) power domain: Contains the PL.
• Baery-power domain: Contains the real-me clock (RTC) as well as baery-backed RAM
(BBRAM).
Other power domains listed in the following gure are not acvely managed by the power
framework. Designs that want to take advantage of the Power Management switching of power
domains must keep some power rails discrete. This allows individual rails to be powered o with
the power domain switching logic. For more details, see the “PCB Power Distribuon and
Migraon in UltraScale+ FPGAs” in the UltraScale Architecture PCB Design User Guide (UG583).
The following diagram illustrates the Zynq UltraScale+ MPSoC power domains and islands.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 170
Figure 50: Zynq UltraScale+ MPSoC Power Domain and Islands
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 171
Because of the heterogeneous mul-core architecture of the Zynq UltraScale+ MPSoC, no single
processor can make autonomous decisions about power states of individual components or
subsystems.
Instead, a collaborave approach is taken, where a power management API delegates all power
management control to the plaorm management unit (PMU). It is the key component
coordinang the power management requests received from the other processing units (PUs),
such as the APU or the RPU, and the coordinaon and execuon from other processing units
through the power management API.
IMPORTANT! In the EEMI implementaon for Zynq UltraScale+ MPSoC, the plaorm management unit
(PMU) serves as the power management controller for the dierent processor units (PUs), such as the APU
and the RPU. These APU/RPU act as a power management (PM) master node and make power
management requests. Based on those requests, the PMU controls the power states of all PM slave nodes
as well as the PM masters. Unless otherwise specied, the terms "PMU" and "power management
controller" are interchangeable.
The Zynq UltraScale+ MPSoC also supports inter-processor interrupts (IPIs), which are used as
the basis for power-management related communicaon between the dierent processors. See
this link to the “Interrupts” chapter of the Zynq UltraScale+ Device Technical Reference Manual
(UG1085) for more detail on this topic.
Zynq UltraScale+ MPSoC Power Management
Software Architecture
To enable mulple processing units to cooperate in terms of power management, the soware
framework for the Zynq UltraScale+ MPSoC provides an implementaon of the power
management API for managing heterogeneous mulprocessing systems.
The following gure illustrates the API-based power management soware architecture.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 172
Figure 51: API-Based Power Management Software Architecture
PMU PMM
APU
OS/Application(s)
PM-API
RPU
RTOS/Application(s)
PM-API
MicroBlaze
Application(s)
PM-API
PM Masters
PM-API
PM Controller
PM Slaves
Memory_A Memory_B
Peripheral_A Peripheral_B
IPI - communication
IPI - communication
Power state
control
Power state control
Power state control
X19503-071317
Power Management Framework Overview
The Zynq UltraScale+ MPSoC power management framework (PMF) is based on an
implementaon of EEMI, see the Embedded Energy Management Interface EEMI API Reference
Guide (UG1200). It includes APIs that consist of funcons available to the processor units (PUs)
to send messages to the power management controller, as well as callback funcons in for the
power management controller to send messages to the PUs. The APIs can be grouped into the
following funconal categories:
• Suspending and waking up PUs
• Slave device power management, such as memories and peripherals
• Miscellaneous
• Direct-access
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 173
API Calls and Responses
Power Management Communication using IPIs
In the Zynq UltraScale+ MPSoC, the power management communicaon layer is implemented
using inter-processor interrupts (IPIs), provided by the IPI block. See this link to the “Interrupts”
chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085) for more details on
IPIs.
Each PU has a dedicated IPI channel with the power management controller, consisng of an
interrupt and a payload buer. The buer passes the API ID and up to ve arguments. The IPI
interrupt to the target triggers the processing of the API, as follows:
• When calling an API funcon, a PU generates an IPI to the power management unit (PMU),
prompng the execuon of necessary power management acon.
• The PMU performs each PM acon atomically, meaning that the acon cannot be interrupted.
• To support PM callbacks, which are used for nocaons from the PMU to a PU, each PU
implements handling of these callback IPIs.
Acknowledge Mechanism
The Zynq UltraScale+ MPSoC power management framework (PMF) supports blocking and non-
blocking acknowledges. In most API calls that oer an acknowledge argument, the caller can
choose one of the following three acknowledge opons:
• REQUEST_ACK_NO: No acknowledge requested
• REQUEST_ACK_BLOCKING: Blocking acknowledge requested
• REQUEST_ACK_NON_BLOCKING: Non-blocking acknowledge using callback requested
Mulple power management API calls are serialized because each processor unit (PU) uses a
single IPI channel for the API calls. Aer one request is sent to the power management
controller, the next one can be issued only aer the power management controller has
completed servicing the rst one. Therefore, no maer which acknowledge mechanism is used,
the caller can be blocked when issuing subsequent requests.
No Acknowledge
If no acknowledge is requested (REQUEST_ACK_NO), the power management controller
processes the request without returning an acknowledge to the caller, otherwise an
acknowledgment is sent.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 174
Blocking Acknowledge
Aer iniang a PM request with the (REQUEST_ACK_BLOCKING) specied, a caller remains
blocked as long as the power management controller does not provide the acknowledgment.
The plaorm management unit (PMU) writes the acknowledge values into the response poron
of the IPI buer before it clears the IPI interrupt. The caller reads the acknowledge values from
the IPI buer aer the IPI observaon register shows that the interrupt is cleared, which is when
PMU has completed servicing the issued IPI. The IPI for the PU is disabled unl the PMU is ready
to handle the next request.
Non-Blocking Acknowledge
Aer iniang a PM request with the (REQUEST_ACK_NON_BLOCKING) specied, a caller does
not wait for the plaorm management unit (PMU) to process that request. Moreover, the caller is
free to perform some other acvies while waing for the acknowledge from the PMU.
Aer the PMU completes servicing the request, it writes the acknowledge values into the IPI
buer. Next, the PMU triggers the IPI to the caller PU to interrupt its acvies, and to inform it
about the sent acknowledge.
Non-blocking acknowledges are implemented using a callback funcon that is implemented by
the calling PU, see XPm_NotifyCb Callback.
For more informaon about XPm_NotifyCb, see XilPM Library in the OS and Libraries Document
Collecon (UG643).
Power Management Framework Layers
There are dierent API layers in the power management framework (PMF) implementaon for
Zynq UltraScale+ MPSoCs, which are, as follows:
• Xilpm: This is a library layer used for standalone applicaons in the dierent processing units,
such as the APU and RPU.
• TF-A: The Trusted Firmware-A (TF-A) contains its own implementaon of the client-side PM
framework. It is currently used by Linux operang systems.
• PMU rmware: The power management unit rmware (PMUFW) runs on the power
management unit (PMU) and implements of the power management API.
For more details, see this link in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
The following gure shows the interacon between the APU, the RPU, and the PMF APIs.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 175
Figure 52: API Layers Used with Bare-Metal Applications Only
RPU
Bare metal application
xilpm
PM-API
PMU
PM-API
APU
Bare metal application
xilpm
PM-API
IPI
X19094-071317
If the APU is running a complete soware stack with an operang system, the Xilpm library is not
used. Instead, the TF-A running on EL3 implements the client-side power management API, and
provides a secure monitor call (SMC)-based interface to the upper layers.
The following gure illustrates this behavior. See the Armv8 manuals for more details on the
Armv8 architecture and its dierent execuon modes. It illustrates the PMF layers that are
involved when running a full soware stack on the APU.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 176
Figure 53: PM Framework Layers Involved When Running a Full Software Stack on the
APU
APU
RPU
OS
ATF
PM-API
EL0/1
EL3
SMC
Bare metal application
xilpm
PM-API
PMU
PM-API
IPI
X19093-071317
Typical Power Management API Call Flow
Any enty involved in power management is referred to as a node. The following secons
describe how the power management framework (PMF) works with slave nodes allocated to the
APU and the RPU.
Generally, the APU or the RPU inform the power management controller about their usage of a
slave node, by requesng for it. They then inform the power management controller about the
capability requirement needed from the slave node. At this point, the power management
controller powers up the slave node so that it can be inialized by the APU or the RPU.
Requesting and Releasing Slave Nodes
When a PU requires a slave node, either peripheral or memory, it must request that slave node
using the power management API. Aer the slave node has performed its funcon and is no
longer required, it may be released, allowing the slave node to be powered o.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 177
The following gure shows the call ow for a use-case in which the APU and the RPU are sharing
an OCM memory bank, ocm0.
Figure 54: PM Framework Call Sequence for APU and RPU Sharing an OCM Memory
Bank
APU
pm_request_node
(nodelD=ocm0,
cap=full, ack=1)
PMC + PSM RPU ocm0
pm_release_node
(node=ocm0, latency=0)
pm_self_suspend
(nodelD=APU 0,
latency=MAX)
WFI interrupt
pm_request_node
(nodeID=ocm0,
cap=full, ack=1)
pm_release_node
(node=ocm0, latency=0)
RUN
POWER
DOWN
RUN RUN OFF
ON
OFF
X20022-111020
Note: The ocm0 memory remains powered on aer the APU calls XStatus XPm_ReleaseNode, because
the RPU has also requested the same slave node. It is aer the RPU also releases the ocm0 node that the
PMU powers o the ocm0 memory.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 178
Processor Unit Suspend and Resume
To allow a processor unit (PU) to be powered o, as opposed to just entering an idle state, an
external enty is required to take care of the power-down and power-up transions. For the
Zynq UltraScale+ MPSoC, the plaorm management unit (PMU) is the responsible enty for
performing all power state changes.
The processor unit (PU) noes the PMU that a power state transion is being requested. The
following gure illustrates the process.
Figure 55: APU Suspend and Resume Procedure
PMU OCM DDR L2$
RUN ON
APU_0
APU_1/2/
3
Peripheral
ON ON RUN
POWER
DOWN
ON
save context
save context
self_suspend
(nodeID-APU_0, latency=MAX)
configure
set_wakeup_source
(targetID=APU, nodeID=Periheral, enable=true)
set_requirement
(nodeID=OCM, capabilities=context, ack=0)
WFI Interrupt
Interrupt
restore context
restore context
serve interrupt
ON ON RUN
Retention
Power
Down
Power
Down
X20023-110217
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 179
The Self-Suspending a CPU/PU secon in Implemenng Power Management on a Processor Unit
provides more details on the suspend or resume procedure. Each PU usually depends on a
number of slave nodes to be able to operate.
Subsystem Power Management
Isolation Configuration
The Zynq UltraScale+ MPSoC can be paroned into sub-systems, so that they can be managed
independently by the power management framework. For example, you can dene a Linux sub-
system and a Real-me sub-system. The Linux sub-system may include the APU (as the PM
master) and a number of peripherals (as the PM slaves). The Real-me sub-system may include
the RPU and a number of other peripherals. Each sub-system can be powered up, powered
down, restarted or suspended without aecng the other sub-systems. A sub-system has only
one PM Master, and may include both FPD and LPD peripherals.
You can create your own sub-systems using the Vivado
®
PCW tool. The following gure shows
the PCW screen shots of a valid conguraon, which contains only an APU sub-system and no
RPU sub-systems.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 180
Figure 56: PCW Configuration
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 181
Figure 57: PCW Configuration Contd
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 182
Figure 58: PCW Configuration Contd
Note: The PCW tool is also used to isolate some peripherals from each other for security purposes. See
Zynq UltraScale+ MPSoC: Embedded Design Tutorial (UG1209) and Zynq UltraScale+ MPSoC Processing System
LogiCORE IP Product Guide (PG201) for details on how to set up isolaon between peripherals.
Configuration Object
The sub-system conguraon is captured in a Conguraon Object, which is generated by the
Vivado and PetaLinux toolchain. The Conguraon Object contains:
• The PM Masters that are present in the system (APU and/or RPU). Any PM Master not
specied in the Conguraon Object will be powered down by the PMU.
• Congurable permissions for each PM Master, such as:
○ Which PM Master can use which PM Slave (A PM Master can use all the PM Slaves that
belong in the same sub-system.)
○ Access to MMIO address regions.
○ Access to peripheral reset lines.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 183
• Pre-allocated PM Slaves. The PM Master can use these PM Slaves without requesng for
them rst. These PM Slaves are needed by the PM Master in order to boot. The toolchain
makes sure that the APU can access the L2 cache and DDR banks without rst requesng for
them. The same is true for the RPU accessing all the TCM banks.
During boot, the Conguraon Object is passed from the FSBL to the PMU rmware. For more
details, see the Conguraon Object.
Note: Isolaon is not required for the Conguraon Object to be created. You can create subsystems to
customize the Conguraon Object and then uncheck the isolaon checkbox.
Power Management Initialization
Power management is disabled during boot and all the peripherals are powered up at this me.
That is because it is oen necessary to allow for possible, and temporary, inter-dependencies
between peripherals during boot and inializaon. When FSBL is nished with inializing the
peripherals and loading the applicaon binaries, it passes the Conguraon Object to the PMU.
The PMU is now aware of all the sub-systems and their associated PM Masters and PM Slaves.
PM Masters and PM Slaves that are not included in the Conguraon Object are never used, and
are powered down by the PMU.
A PM Master is not likely to use all the PM Slaves at all mes. Therefore, a PM Slave should be
powered up only when it is being used. The PM Master must nofy the PMU before and aer
using a PM Slave. This funconality is implemented in the PetaLinux kernel. This requirement
hinders developers starng with a new RPU applicaon, when the focus is on funconality and
not power opmizaon. Therefore, it is convenient for the PMU to also support PM-incapable
Masters that do not provide nocaons when they are using the PM Slaves. This is done by
keeping all the PM Slaves in the sub-system powered up unl the PM Master sends the
PmInitFinalize request to the PMU. A PM-incapable Master will never send this request,
which means that its PM Slaves will remain powered up at all mes or unl this PM Master itself
is powered down.
A PM-capable Master sends this request aer inializing the sub-system. The PMU then begins
powering down the PM Slaves in this sub-system whenever they are not being used.
As a result, when there is an RPU master present in the system but it is not running any
applicaon, the PMU rmware will consider it as a PM incapable master and hence will never
power down the RPU and its slaves. From the 2018.3 release and onwards, this behavior is xed
and allows you to power down unused RPUs. This change is protected by the compilaon ag
ENABLE_UNUSED_RPU_PWR_DWN and is enabled by default. When this ag is enabled, the
unused RPU and allocated slaves will be powered down if not in use.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 184
Table 56: Power Management Initialization
Boot Mode APU Image RPU Image RPU Mode
ENABLE_UN
USED_RPU_P
WR_DWN_VA
L
RPU0/TCM0
State
RPU1/ TCM1
State
JTAG Don’t Care Don’t Care Don’t Care Don’t Care ON ON
Non-JTAG Boot
modes
Don’t Care Don’t Care Don’t Care 0 ON ON
Linux or PM
aware app that
calls
PM_INIT_FINALI
ZE
RPU app in
boot image
Lock step mode 1 ON OFF
RPU0 app in
boot image
Split mode 1 ON OFF
RPU1 app in
boot image
Split mode 1 OFF ON
Both RPU apps
in boot image
Split mode 1 ON ON
RPU app not in
boot image
1 OFF OFF
Bare Metal app
that does not
call
PM_INIT_FINALI
ZE
Don’t Care
Don’t Care Don’t Care ON ON
Note: If you do not want to power down RPU/TCM by default, set the
ENABLE_UNUSED_RPU_PWR_DWN ag to 0 while compiling the PMU rmware. For the JTAG boot
mode there is no impact on behavior change even though ENABLE_UNUSED_RPU_PWR_DWN ag is 1.
Note: Sub-systems may overlap each other. This means that some PM Slaves may belong to more than one
sub-system (for example, DDR, OCM, and so on). If a PM Slave is in more than one sub-system, the PMU
does not power down this PM Slave unl it has been released by all its PM Masters, or unl all these PM
Masters have powered down themselves.
Default Configuration
By default, Isolaon Conguraon is disabled, and the tool chain generates a conguraon with
three sub-systems. Each has a PM Master: APU, R5-0 and R5-1. All three sub-systems contain all
the PM Slaves (meaning that the sub-systems completely overlap each other.) This is the default
conguraon generated by PCW when the “Enable Isolaon” box is unchecked. The default
PetaLinux kernel conguraon is PM-capable, but R5-0 and R5-1 must be also running “PM-
capable” applicaons, or be powered down. Otherwise, the PMU will not power down any PM
Slaves.
Note: You can create a conguraon that does not allow the processors to boot and run. If you are a
beginner, use the APU-only conguraon as described in Isolaon Conguraon secon and customize it
as necessary.
RPU Lock-step vs. Split Mode
The toolchain infers the RPU run modes from the PCW Isolaon Conguraon as follows:
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 185
• No RPU present in any subsystem: Conguraon Object contains no RPU.
• Only R5-0 present in subsystem(s): Conguraon Object contains R5-0 running in lock-step
mode.
• Both R5-0 and R5-1 in subsystems: Conguraon Object contains R5-0 and R5-1 running in
split mode.
• Only R5-1 present in subsystem(s): Conguraon Object contains R5-1 running in split mode.
The default Conguraon Object contains two RPU PM Masters: R5-0 and R5-1, and the PMU
assumes that the R5-0 and R5-1 are running in split mode. However, the boot image actually
determines whether the RPU runs in lock-step or split mode at boot me. The RPU run mode
from the boot image must match the number of RPU PM Masters in the Conguraon Object.
Otherwise, the power management framework will not work properly.
Note: If you intend to use the R5 in lock-step mode, you need to ensure that the Isolaon Conguraon is
enabled in PCW, and only R5-0 (not R5-1) is present in a subsystem.
Sharing Devices
Sharing access to devices between APU and RPU is possible but must always be done with great
care. The access and operaon of a device depend on its clock (if applicable), its conguraon
and its power state (on, o, retenon, and so on.) The PMU makes sure the device is in the
lowest power state that will sasfy the requirement of all the PM Masters, but it is up to the APU
and RPU to set up the clock and conguraon of the device.
Extra care must be taken when a device is shared between the APU running Linux and the RPU.
Linux is not aware that another enty might be using one of its devices, and will clock-gate,
power-gate and disable the device whenever it is not being used. The opons available are:
• Disable Linux runme power management of the device. See hps://www.kernel.org/doc/
Documentaon/ABI/tesng/sysfs-devices-power. This will keep the device running even
when Linux is not using it, but the device will sll be clock-gated and disabled when Linux
goes to sleep.
• Implement a special driver for the device.
Any devices not used by the APU should be removed from the device tree.
Using the API for Power Management
This chapter contains detailed instrucons on how to use the Xilinx
®
power management
framework (PMF) APIs to carry out common power management tasks.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 186
Implementing Power Management on a Processor
Unit
The Xilpm library provides the funcons that the standalone applicaons execung on a
processor can use to iniate the power management API calls.
See the SDK Online Help (UG782) for informaon on how to include the Xilpm library in a project.
Initializing the Xilpm Library
Before iniang any power management API calls, you must inialize the Xilpm library by calling
XPm_InitXilpm, and passing a pointer to a properly inialized inter-processor interrupt (IPI)
driver instance.
See this link to the “Interrupts” chapter of the Zynq UltraScale+ Device Technical Reference Manual
(UG1085). for more informaon regarding IPIs.
Working with Slave Devices
The Zynq UltraScale+ MPSoC power management framework (PMF) contains funcons
dedicated to managing slave devices (also referred to as PM slaves), such as memories and
peripherals. Processor units (PUs) use these funcons to inform the power management
controller about the requirements (such as capabilies and wake-up latencies) for those devices.
The power management controller manages the system so that each device resides in the lowest
possible power state, meeng the requirements from all eligible PUs.
Requesting and Releasing a Node
A PU uses the XPm_RequestNode API to request the access to a slave device and assert its
requirements on that device. The power management controller manages the requested device's
power-on and acve state, provided the PU and the slave belong to the same sub-system.
Aer a device is no longer used, the PU typically calls the XPm_ReleaseNode funcon to allow
the PM controller to re-evaluate the power state of that device, and potenally place it into a
low-power state. It also then allows other PUs to request that device.
Changing Requirements
When a PU is using a PM slave, its requirement on the slave's capability may change. For
example, an interface port may go into a low power state, or even be completely powered o, if
the interface is not being used. The PU may use XPm_SetRequirement to change the
capability requirement of the PM slave. Typically, the PU would not release the PM slave if it will
be changing the requirement again in the future.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 187
The following example call changes the requirement for the node argument to require wake-
interrupts only:
XPm_SetRequirement(node, PM_CAP_WAKEUP, 0, REQUEST_ACK_NO);
IMPORTANT! Seng requirements of a node to zero is not equivalent to releasing the PM slave. By
releasing the PM slave, a PU may be allowing other PUs to use the device exclusively.
When mulple PUs share a PM slave (this applies mostly to memories), the power management
controller selects a power state of the PM slave that sases all requirements of the requesng
PUs.
The requirements on a PM slave include capability as well as latency requirements. Capability
requirements may include a top capability state, some intermediate capability states, an inacve
state (but with the conguraon retained), and the o state. Latency requirement species the
maximum me allowed for the PM slave to switch to the top capability state from any other
state. If this me limit cannot be met, the power management controller will leave the PM slave
in the top capability state regardless of other capability requirements.
Self-Suspending a CPU/PU
A PU can be a cluster of CPUs. The APU is a PU, that has four CPUs. An RPU has two CPUs, but
it is considered as two PUs when running in the split mode, and one PU when it is running in the
lock-step mode.
To suspend itself, a CPU must inform the power management controller about its intent by calling
the XPM_SelfSuspend funcon. The following acons then occur:
• Aer the XPm_SelfSuspend() call is processed, none of the future interrupts can prevent
the CPU from entering a sleep state. To manage such behavior in the case of the APU and
RPU, aer the XPm_SelfSuspend() call has completed, all of the interrupts to a CPU are
directed to the power management controller as GIC wake interrupts.
• The power management controller then waits for the CPU to nalize the suspend procedure.
The PU informs the power management controller that it is ready to enter a sleep state by
calling XPm_SuspendFinalize.
• The XPm_SuspendFinalize() funcon is architecture-dependent. It ensures that any
outstanding power management API call is processed, then executes the architecture-specic
suspend sequence, which also signals the suspend compleon to the power management
controller.
• For Arm
®
processors such as the APU and RPU, the XPm_SuspendFinalize() funcon
uses the wait for interrupt (WFI) instrucon, which suspends the CPU and triggers an
interrupt to the power management controller.
• When the suspend compleon is signaled to the power management controller, the power
management controller places the CPU into reset, and may power down the power island of
the CPU, provided that no other component within the island is currently acve.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 188
• Interrupts enabled through the GIC interface of the CPU are redirected to the power
management controller (PMC) as a GIC wake interrupt assigned to that parcular CPU.
Because the interrupts are redirected, the CPU can only be woken up using the power
management controller.
• Suspending a PU requires suspending all of its CPUs individually.
Resuming Execution
A CPU can be woken up either by a wake interrupt triggered by a hardware resource or by an
explicit wake request using the XPM_RequestWakeup API.
The CPU starts execung from the resume address provided with the XPm_SelfSuspend call.
Setting up a Wake-up Source
The power management controller can power down the enre FPD if none of the FPD devices
are in use and exisng latency requirements allow this acon. If the FPD is powered o and the
APU is to be woken up by an interrupt triggered by a device in the LPD, the GIC Proxy must be
congured to allow propagaon of FPD wake events. The APU can ensure this by calling
XPM_SetWakeUpSource for all devices that might need to issue wake interrupts.
Hence, prior to suspending, the APU must call XPm_SetWakeupSource(NODE_APU, node,
1) to add the required slaves as a wake-up source. The APU can then set the requirements to
zero for all slaves it is using. Aer the APU nalizes its suspend procedure, and provided that no
other PU is using any resource in the FPD, the PM controller powers o the enre FPD and
congures the GIC proxy to enable propagaon of the wake event of the LPD slaves.
Aborting a Suspend Procedure
If a PU decides to abort the suspend procedure aer calling the XPM_SetSelfSuspend
funcon, it must inform the power management controller about the aborted suspend by calling
the XPm_AbortSuspend funcon.
Handling PM Slaves During the Suspend Procedure
A PU that suspends itself must inform the power management controller about its changed
requirements on the peripherals and memories in use. If a PU fails inform the power management
controller, all of the used devices remain powered on. Typically, for memories you must ensure
that their context is preserved by using the following funcon:
XPm_SetRequirement(node, PM_CAP_CONTEXT, 0, REQUEST_ACK_NO);
When seng requirements for a PM slave during the suspend procedure; aer calling
XPM_SelfSuspend, the seng is deferred unl the CPU nishes the suspend. This deference
ensures that devices that are needed for compleng the suspend procedure can enter a low
power state aer the calling CPU nishes suspend.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 189
A common example is instrucon memory, which a CPU can access unl the end of a suspend.
Aer the CPU suspends a memory, that memory can be placed into retenon. All deferred
requirements reverse automacally before the respecve CPU is woken up.
When an enre PU suspends, the last awake CPU within the PU must manage the changes to the
devices.
Example Code for Suspending an APU/RPU
There the following is an example of source code for suspending the APU or RPU:
/* Base address of vector table (reset-vector) */ extern void
*_vector_table;
/* Inform PM controller that APU_0 intends to suspend */
XPm_SelfSuspend(NODE_APU_0, MAX_LATENCY, 0, (u64)&_vector_table);
/**
* Set requirements for OCM banks to preserve their context.
* The PM controller will defer putting OCMs into retention until the
suspend is finalized
*/
XPm_SetRequirement(NODE_OCM_BANK_0, PM_CAP_CONTEXT, 0, REQUEST_ACK_NO);
XPm_SetRequirement(NODE_OCM_BANK_1, PM_CAP_CONTEXT, 0, REQUEST_ACK_NO);
XPm_SetRequirement(NODE_OCM_BANK_2, PM_CAP_CONTEXT, 0, REQUEST_ACK_NO);
XPm_SetRequirement(NODE_OCM_BANK_3, PM_CAP_CONTEXT, 0, REQUEST_ACK_NO);
/* Flush data cache */ Xil_DCacheFlush();
/* Inform PM controller that suspend procedure is completed */
XPm_SuspendFinalize();
Suspending the Entire FPD Domain
To power-down the enre full power domain, the power management controller must suspend
the APU at a me when none of the FPD devices is in use. Aer this condion is met, the power
management controller can power-down the FPD automacally. The power management
controller powers down the FPD if no latency requirements constrain this acon, otherwise the
FPD remains powered on.
Forcefully Powering Down the FPD
There is the opon to force the FPD to power-down by calling the funcon
XPM_ForcePowerdown. This requires that the requesng PU has proper privileges congured in
the power management controller. The power management controller releases all PM Slaves
used by the APU automacally.
Note: This force method is typically not recommended, especially when running complex operang systems
on the APU because it could result in loss of data or system corrupon, due to the OS not suspending itself
gracefully.
IMPORTANT!
Use the
XPm_RequestSuspend
  API.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 190
Interacting with Other Processing Units
Suspending a PU
A PU can request that another PU be suspended by calling XPm_RequestSuspend, and passing
the targeted node name as an argument.
This causes the power management controller to call XPm_InitSuspendCb(), which is a
callback funcon implemented in the target PU. The target PU then iniates its own suspend
procedure, or call XPm_AbortSuspend and specify the abort reason. For example, you can
request an APU to suspend with the following command:
XPm_RequestSuspend(NODE_APU, REQUEST_ACK_NON_BLOCKING, MAX_LATENCY, 0);
The following diagram shows the general sequence triggered by a call to the
XPM_RequestSuspend.
For more informaon about XPm_RequestSuspend, XPm_InitSuspendCb, and
XPm_AbortSuspend, see XilPM Library in the OS and Libraries Document Collecon (UG643).
Figure 59: APU initiating suspend for the RPU by calling XPm_RequestSuspend
APU
XPm_RequestSuspend
(nodeID=RPU,
latency=MAX)
PMU RPU TCM
RUN RUN RUN ON
XPm_InitSuspendCb
(reason=PU REQ,
latency=MAX)
XPm_SelfSuspend
(nodeID=RPU,
latency=MAX)
XPm_SetRequirement
(nodeID=TCM,
cap=context, ack=0)
XPm_SuspendFinalize
Power
Down
Retention
save context
X20024-110217
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 191
Waking a PU
Addionally, a PU can request the wake-up of one of its CPUs or of another PU by calling
XPm_RequestWakeup.
• When processing the call, the power management controller causes a target CPU or PU to be
awakened.
• If a PU is the target, only one of its CPUs is woken-up by this request.
• The CPU chosen by the power management controller is considered the primary CPU within
the PU.
The following is an example of a wake-up request:
XPm_RequestWakeup(NODE_APU_1, REQUEST_ACK_NO);
For more informaon about XPm_RequestWakeup, see XilPM Library in the OS and Libraries
Document Collecon (UG643).
DDR Self-refresh over Warm Restart
In most systems, the RAM of a compung system is cleared when the system resets or powers
down. Any data that needs to be retained, such as sengs and logs, are usually stored in a non-
volale memory such as ash and baery backed-up RAM. These non-volale memories are
slower, especially when the amount of data is huge. For some systems, a more preferred soluon
is to retain the data in the DRAM, thus eecvely using it as a non-volale memory.
The Zynq UltraScale+ MPSoC soware soluon supports a feature to put DDR into self-refresh
mode during warm restart (system reset, or PS only reset). This makes the DDR a non-volale
memory and its contents remain as it is even aer a reset.
By default, this feature is disabled. You can enable this feature by enabling the following build
ags during PMUFW and FSBL compilaon:
• PMUFW: ENABLE_DDR_SR_WR
• FSBL: XFSBL_ENABLE_DDR_SR
Aer these build ags are enabled, the PMUFW puts the DDR in self-refresh mode during a
warm restart (PS only or System restart).
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 192
XilPM Implementation Details
The system layer of the PM framework is implemented on the Zynq UltraScale+ MPSoC using
inter-processor interrupts (IPIs). To issue an EEMI API call, a PU will write the API data (API ID
and arguments) into the IPI request buer and then trigger the IPI to the PMU.
Aer the PM controller processes the request it will send the acknowledge depending on the
parcular EEMI API and provided arguments.
Payload Mapping for API Calls to PMU
Each EEMI API call is uniquely idened by the following data:
• EEMI API idener (ID)
• EEMI API arguments
Please see Appendix A for a list of all API ideners as well as API argument values.
Prior to iniang an IPI to the PMU, the PU shall write the informaon about the call into the IPI
request buer. Each data wrien into the IPI buer is a 32-bit word. Total size of the payload is
six 32-bit words - one word is reserved for the EEMI API idener, while the remaining words
are used for the arguments. Wring to the IPI buer starts from oset zero. The informaon is
mapped as follows:
• Word [0] EEMI API ID
• Word [1:5] EEMI API arguments
The IPI response buer is used to return the status of the operaon as well as up to 3 values.
• Word [0] success or error code
• Word [1:3] value 1..3
Payload Mapping for API Callbacks from the PMU
The EEMI API includes callback funcons, invoked by the PM controller, sent to a PU.
• Word [0]EEMI API Callback ID
• Word [1:5]EEMI API arguments
Refer to the XilPM Library in the OS and Libraries Document Collecon (UG643) for a list of all API
ideners as well as API argument values.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 193
Issuing EEMI API calls to the PMU
Before issuing an API call to the PMU, a PU must wait unl its previous API call is processed by
the PMU. A check for compleon of a PMU acon can be implemented by reading the
corresponding IPI observaon register.
An API call is issued by populang the IPI payload buer with API data and triggering an IPI
interrupt to the PMU. In case of a blocking API call, the PMU will respond by populang the
response buer with the status of the operaon and up to 3 values. See Appendix B: Addional
Resources and Legal Noces for a list of all the errors that can be sent by the PMU if a PM
operaon was unsuccessful. The PU must wait unl the PMU has nished processing the API call
prior to reading the response buer, to ensure that the data in the response buer is valid.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 194
Figure 60: Example Flow of Issuing API Call to the PMU
XEMI API Call
Previous
API call
processed by
the PMU?
Copy API data into IPI
request buffer
Trigger IPI interrupt
Blocking API call?
API call
processed by
the PMU?
Read response from
the PMU
Return
No
No
Yes
Yes
Yes
X19506-071017
Handling API callbacks from the PMU
The PMU invokes callback funcons to the PU by populang the IPI buers with the API callback
data and triggering an IPI interrupt to the PU. In order to receive such interrupts, the PU must
properly inialize the IPI block and interrupt controller. A single interrupt is dedicated to all
callbacks. For this reason, element 0 of the payload buer contains the API ID, which the PU
should use to idenfy the API callback. The PU should then call the respecve API callback
funcon, passing in the arguments obtained from locaons 1 to 4 of the IPI request buer.
An implementaon of this behavior can be found in the XilPM library.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 195
Linux
Linux executes on the EL1 level, and the communicaon between Linux and the TF-A soware
layer is realized using SMC calls.
Power management features based on the EEMI API have been ported to the Linux kernel,
ensuring that the Linux-centric power management features ulize the EEMI services provided
by the PMU.
Addionally, the EEMI API can be access directly through debugfs for debugging purposes. Note
that direct access to the EEMI API through debugfs will interfere with the kernel power
management operaons and may cause unexpected problems.
All the Linux power management features presented in this chapter are available in the PetaLinux
default conguraon.
User Space PM Interface
System Power States
You may request to change the power state of a system or the enre system. The PMU facilitates
the switching of the system or sub-system to the new power state.
Shutdown
You may shutdown the APU sub-system with the standard 'shutdown' command.
To shut down the enre system, the user must shut down all the other sub-systems prior to
shung down the APU sub-system. For example, use the following command to power down
the PL.
echo pm_release_node 69 > /sys/kernel/debug/zynqmp-firmware/pm
Use this command to power up the PL again:
echo pm_request_node 69 > /sys/kernel/debug/zynqmp-firmware/pm
For informaon about how to shut down the PL sub-system, see the Libmetal and OpenAMP for
Zynq Devices User Guide (UG1186).
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 196
Reboot
You can use the reboot command to reset the APU, the PS or the System. By default, the reboot
command resets the system. You can change the scope of the reboot command to APU or PS if
required. To change the reboot scope to APU:
echo subsystem > /sys/firmware/zynqmp/shutdown_scope
To change the reboot scope to PS:
echo ps_only > /sys/firmware/zynqmp/shutdown_scope
To change the reboot scope to System:
echo system > /sys/firmware/zynqmp/shutdown_scope
The reboot scope is set to System again aer the reset.
Suspend
The kernel is suspended when the CPU and most of the peripherals are powered down. The
system run states needed to resume from suspend is stored in the DRAM, which is put into self-
refresh mode.
Kernel conguraons required:
• Power management opons
○ [*] Suspend to RAM and standby
○ [*] User space wakeup sources interface
○ [*] Device power management core funconality
• Device Drivers
○ System-on-a-chip (SoC) specic Drivers
- Xilinx
®
SoC drivers
- Zynq MPSoC SoC
- [*] Enable Xilinx Zynq MPSoC Power Management driver
- [*] Enable Zynq MPSoC generic PM domains
• Firmware Drivers
○ Zynq MPSoC Firmware Drivers
- [*] Enable Xilinx Zynq MPSoC rmware interface
Note: Any device can prevent the kernel from suspending.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 197
See also hps://wiki.archlinux.org/index.php/Power_management/Suspend_and_hibernate.
To suspend the kernel:
$ echo mem > /sys/power/state
Wake-up Source
The kernel resumes from the suspend mode when a wake-up event occurs. The following wake-
up sources can be used:
• UART
If enabled as a wake-up source, a UART input will trigger the kernel to resume from the
suspend mode.
Kernel conguraons required:
○ Same as Suspend.
For example, to wake up the APU on UART input:
$ echo enabled > /sys/devices/platform/amba/ff000000.serial/tty/ttyPS0/
power/wakeup
• RTC
If enabled as a wake-up source, the kernel will resume from the suspend mode when the RTC
mer expires. Note that the RTC wake-up source is enabled by default.
Kernel conguraons required:
○ Same as Suspend.
For example, to set RTC to wake up the APU aer 10 seconds:
$ echo +10 > /sys/class/rtc/rtc0/wakealarm
• GPIO
If enabled as a wake-up source, a GPIO event will trigger the kernel to resume from the
suspend mode.
Kernel conguraons required:
○ Device Drivers
- Input device support, [*]
Generic input layer (needed for keyboard, mouse, ...) (INPUT [=y]) [*] Keyboards
(INPUT_KEYBOARD [=y])
[*] GPIO Buons (CONFIG_KEYBOARD_GPIO=y)
[*] Polled GPIO buons
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 198
For example, to wake up the APU on the GPIO pin:
$ echo enabled > /sys/devices/platform/gpio-keys/power/wakeup
Power Management for the CPU
CPU Hotplug
The user may take one or more APU cores on-line and o-line as needed through the CPU
Hotplug control interface.
Kernel conguraons required:
• Kernel Features
○ [*] Support for hot-pluggable CPUs
See also:
• hps://www.kernel.org/doc/Documentaon/cpu-hotplug.txt
• hp://lxr.free-electrons.com/source/Documentaon/devicetree/bindings/arm/idle-states.txt
For example, to take CPU3 o-line:
$ echo 0 > /sys/devices/system/cpu/cpu3/online
CPU Idle
If enabled, the kernel may cut power to individual APU cores when they are idling. The kernel
conguraons required are:
• CPU Power Management
○ CPU Idle
- [*] CPU idle PM support
- Arm CPU Idle Drivers
- [*] Generic Arm/Arm64 CPU idle Driver
See also:
• hps://www.kernel.org/doc/Documentaon/cpuidle/core.txt
• hps://www.kernel.org/doc/Documentaon/cpuidle/driver.txt
• hps://www.kernel.org/doc/Documentaon/cpuidle/governor.txt
• hps://www.kernel.org/doc/Documentaon/cpuidle/sysfs.txt
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 199
Below is the sysfs interface for cpuidle.
$ ls -lR /sys/devices/system/cpu/cpu0/cpuidle/
/sys/devices/system/cpu/cpu0/cpuidle/:
drwxr-xr-x 2 root root 0 Jun 10 21:55 state0
drwxr-xr-x 2 root root 0 Jun 10 21:55 state1
/sys/devices/system/cpu/cpu0/cpuidle/state0:
-r--r--r-- 1 root root 4096 Jun 10 21:55 desc
-rw-r--r-- 1 root root 4096 Jun 10 21:55 disable
-r--r--r-- 1 root root 4096 Jun 10 21:55 latency
-r--r--r-- 1 root root 4096 Jun 10 21:55 name
-r--r--r-- 1 root root 4096 Jun 10 21:55 power
-r--r--r-- 1 root root 4096 Jun 10 21:55 residency
-r--r--r-- 1 root root 4096 Jun 10 21:55 time
-r--r--r-- 1 root root 4096 Jun 10 21:55 usage
/sys/devices/system/cpu/cpu0/cpuidle/state1:
-r--r--r-- 1 root root 4096 Jun 10 21:55 desc
-rw-r--r-- 1 root root 4096 Jun 10 21:55 disable
-r--r--r-- 1 root root 4096 Jun 10 21:55 latency
-r--r--r-- 1 root root 4096 Jun 10 21:55 name
-r--r--r-- 1 root root 4096 Jun 10 21:55 power
-r--r--r-- 1 root root 4096 Jun 10 21:55 residency
-r--r--r-- 1 root root 4096 Jun 10 21:55 time
-r--r--r-- 1 root root 4096 Jun 10 21:55 usage
where:
• desc: Small descripon about the idle state (string)
• disable: Opon to disable this idle state (bool)
• latency: Latency to exit out of this idle state (in microseconds)
• name: Name of the idle state (string)
• power: Power consumed while in this idle state (in milliwas)
• me: Total me spent in this idle state (in microseconds)
• usage: Number of mes this state was entered (count)
Below is the sysfs interface for cpuidle governors.
$ ls -lR /sys/devices/system/cpu/cpuidle/
/sys/devices/system/cpu/cpuidle/:
-r--r--r-- 1 root root 4096 Jun 10 21:55 current_driver
-r--r--r-- 1 root root 4096 Jun 10 21:55 current_governor_ro
CPU Frequency
If enabled, the CPU cores may switch between dierent operaon clock frequencies. The kernel
conguraons required are:
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 200
• CPU Frequency scaling
○ [*] CPU Frequency scaling
○ Default CPUFreq governor
- Userspace
• CPU Power Management
○ [*] CPU Frequency scaling
○
Default CPUFreq governor
- Userspace
- <*> Generic DT based cpufreq driver
Look up the available CPU speeds:
$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_cpu_freq
Select the 'userspace' governor for CPU frequency control:
$ echo userspace > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
Look up the current CPU speed (same for all cores):
$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_cpu_freq
Change the CPU speed (same for all cores):
$ echo <freq> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed
For details on adding and changing CPU frequencies, see the Linux kernel documentaon on
Generic Operang Points.
Power Management for the Devices
Clock Gating
Stop device clocks when they are not being used (also called Common Clock Framework.) The
kernel conguraons required are:
• Common Clock Framework
○ [*] Support for Xilinx ZynqMP Ultrascale+ clock controllers
Runtime PM
Power o devices when they are not being used. Note that individual drivers may or may not
support run-me power management. The kernel conguraons required are:
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 201
• Power management opons
○ [*] Suspend to RAM and standby
• Device Drivers
○ SoC (System-on-a-chip) specic drivers
- [*] Xilinx SoC drivers
- [*] Enable Xilinx Zynq MPSoC Power Management driver
- [*] Enable Zynq MPSoC generic PM domains
Global General Storage Registers
Four 32-bit storage registers are available for general use. Their values are not preserved across
aer soware reboots. The following table lists the global general storage registers.
Table 57: Global General Storage Registers
Device Node MMIO Register MMIO Address Valid Value Range
/sys/firmware/zynqmp/ggs0 GLOBAL_GEN_STORAGE0 0xFFD80030 0x00000000 - 0xFFFFFFFF
/sys/firmware/zynqmp/ggs1 GLOBAL_GEN_STORAGE1 0xFFD80034 0x00000000 - 0xFFFFFFFF
/sys/firmware/zynqmp/ggs2 GLOBAL_GEN_STORAGE2 0xFFD80038 0x00000000 - 0xFFFFFFFF
/sys/firmware/zynqmp/ggs3 GLOBAL_GEN_STORAGE3 0xFFD8003C 0x00000000 - 0xFFFFFFFF
Read the value of a global storage register:
$cat /sys/firmware/zynqmp/ggs0
Write the mask and value of a global storage register:
$echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware/zynqmp/ggs0
Persistent Global General Storage Registers
Four 32-bit persistent global storage registers are available for general use. Their values are
preserved across aer soware reboots. The lists the persistent global general storage registers.
Table 58: Persistent Global General Storage Registers
Device Node MMIO Register MMIO Address Valid Value Range
/sys/firmware/zynqmp/pggs0 PERS_GLOB_GEN_STORAGE0 0xFFD80050 0x00000000 -0xFFFFFFFF
/sys/firmware/zynqmp/pggs1 PERS_GLOB_GEN_STORAGE1 0xFFD80054 0x00000000 -0xFFFFFFFF
/sys/firmware/zynqmp/pggs2 PERS_GLOB_GEN_STORAGE2 0xFFD80058 0x00000000 -0xFFFFFFFF
/sys/firmware/zynqmp/pggs3 PERS_GLOB_GEN_STORAGE3 0xFFD8005C 0x00000000 -0xFFFFFFFF
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 202
Read the value of a persistent global storage register:
$cat /sys/firmware/zynqmp/pggs0
Write the mask and value of a persistent global storage register:
$echo 0xFFFFFFFF 0x1234ABCD > /sys/firmware/zynqmp/pggs0
Demo
A demo script is included with the PetaLinux pre-built images, which performs a few simple
power management tasks:
• System Suspend
• CPU Hotplug
• CPU Freq
• System Reboot
• System Shutdown
To start the demo, type the following command:
$ hellopm
Debug Interface
The PM plaorm driver exports a standard debugfs interface to access all EEMI services. The
interface is intended for tesng only and does not contain any checking regarding improper
usage, and the number, type and valid ranges of the arguments. The user should be aware that
invoking EEMI services directly via this interface can very easily interfere with the kernel power
management operaons, resulng in unexpected behavior or system crash. Zynq MPSoC debugfs
interface is disabled by default in defcong. It needs to be enabled explicitly as menoned below.
Kernel conguraons required (in this order):
• Kernel hacking
○
Compile-me checks and compiler opons
- [*] Debug File system
• Firmware Drivers
○ Zynq MPSoC Firmware Drivers
- [*] Enable Xilinx Zynq MPSoC rmware interface
- [*] Enable Xilinx Zynq MPSoC rmware debug APIs
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 203
You may invoke any EEMI API except for:
• Self Suspend
• System Shutdown
• Force Power Down the APU
• Request Wake-up the APU
Command-line Input
The user may invoke an EEMI service by wring the EEMI API ID, followed by up to four
arguments, to the debugfs interface node.
API ID
Funcon ID can be EEMI API funcon name or ID number, type string or type integer,
respecvely.
Arguments
The number and type of the arguments directly depend on the selected API funcon. All
arguments must be provided as integer types and represent the ordinal number for that specic
argument type from the EEMI argument list. For more informaon about funcon descripons,
type and number of arguments see the EEMI API Specicaon.
Example
The following example shows how to invoke a request_node API call for NODE_USB_0.
$ echo "pm_request_node 22 1 100 1" > /sys/kernel/debug/zynqmp-firmware/pm
Command List
Get API Version
Get the API version.
$ echo pm_get_api_version > /sys/kernel/debug/zynqmp-firmware/pm
Request Suspend
Request another PU to suspend itself.
$ echo pm_request_suspend <node> > /sys/kernel/debug/zynqmp-firmware/pm
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 204
Self Suspend
Nofy PMU that this PU is about to suspend itself.
$ echo pm_self_suspend <node> > /sys/kernel/debug/zynqmp-firmware/pm
Force Power Down
Force another PU to power down.
$ echo pm_force_powerdown <node> > /sys/kernel/debug/zynqmp-firmware/pm
Abort Suspend
Nofy PMU that the aempt to suspend has been aborted.
$ echo pm_abort_suspend > /sys/kernel/debug/zynqmp-firmware/pm
Request Wake-up
Request another PU to wake up from suspend state.
$ echo pm_request_wakeup <node> <set_address> <address> > /sys/kernel/debug/
zynqmp-firmware/pm
Set Wake-up Source
Set up a node as the wake-up source.
$ echo pm_set_wakeup_source <target> <wkup_node> <enable> > /sys/kernel/
debug/zynqmp-firmware/pm
Request Node
Request to use a node.
$ echo pm_request_node <node> > /sys/kernel/debug/zynqmp-firmware/pm
Release Node
Free a node that is no longer being used.
$ echo pm_release_node <node> > /sys/kernel/debug/zynqmp-firmware/pm
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 205
Set Requirement
Set the power requirement on the node.
$ echo pm_set_requirement <node> <capabilities> > /sys/kernel/debug/zynqmp-
firmware/pm
Set Max Latency
Set the maximum wake-up latency requirement for a node.
$ echo pm_set_max_latency <node> <latency> > /sys/kernel/debug/zynqmp-
firmware/pm
Get Node Status
Get status informaon of a node. (Any PU can check the status of any node, regardless of the
node assignment.)
$ echo pm_get_node_status <node> > /sys/kernel/debug/zynqmp-firmware/pm
Get Operating Characteristic
Get operang characterisc informaon of a node.
$ echo pm_get_operating_characteristic <node> > /sys/kernel/debug/zynqmp-
firmware/pm
Reset Assert
Assert/de-assert on specic reset lines.
$ echo pm_reset_assert <reset> <action> > /sys/kernel/debug/zynqmp-
firmware/pm
Reset Get Status
Get the status of the reset line.
$ echo pm_reset_get_status <reset> > /sys/kernel/debug/zynqmp-firmware/pm
Get Chip ID
Get the chip ID.
$ echo pm_get_chipid > /sys/kernel/debug/zynqmp-firmware/pm
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 206
Get Pin Control Functions
Get current selected funcon for given pin.
$ echo pm_pinctrl_get_function <pin-number> > /sys/kernel/debug/zynqmp-
firmware/pm
Set Pin Control Functions
Set requested funcon for given pin.
$ echo pm_pinctrl_set_function <pin-number> <function-id> > /sys/kernel/
debug/zynqmp-firmware/pm
Get Configuration Parameters for the Pin
Get value of requested conguraon parameter for given pin.
$ echo pm_pinctrl_config_param_get <pin-number> <parameter to get> > /sys/
kernel/debug/zynqmp-firmware/pm
Set Configuration Parameters for the Pin
Set value of requested conguraon parameter for given pin.
$ echo pm_pinctrl_config_param_set <pin-number> <parameter to set> <param
value> > /sys/kernel/debug/zynqmp-firmware/pm
Control Device and Configurations
Control device and conguraons and get conguraons values.
$ echo pm_ioctl <node id> <ioctl id> <arg1> <arg2> > /sys/kernel/debug/
zynqmp-firmware/pm
Table 59: IOCTLs in SDG
IOCTL_ID Name Description
0 IOCTL_GET_RPU_OPER_MODE returns current RPU operating mode (lockstep/split).
1 IOCTL_SET_RPU_OPER_MODE configures RPU operating mode (lockstep/split).
2 IOCTL_RPU_BOOT_ADDR_CONFIG configures RPU boot address
3 IOCTL_TCM_COMB_CONFIG configures TCM to be in split mode or combined
mode
4 IOCTL_SET_TAPDELAY_BYPASS enable/disable tap delay bypass
5 IOCTL_SET_SGMII_MODE enable/disable SGMII mode for the GEM device
6 IOCTL_SD_DLL_RESET resets DLL logic for the SD device
7 IOCTL_SET_SD_TAPDELAY sets input/output tap delay for the SD device
8 IOCTL_SET_PLL_FRAC_MODE sets PLL mode
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 207
Table 59: IOCTLs in SDG (cont'd)
IOCTL_ID Name Description
9 IOCTL_GET_PLL_FRAC_MODE returns current PLL mode
10 IOCTL_SET_PLL_FRAC_DATA sets PLL fraction data
11 IOCTL_GET_PLL_FRAC_DATA returns PLL fraction data value
12 IOCTL_WRITE_GGS writes value to GGS register
13 IOCTL_READ_GGS returns GGS register value
14 IOCTL_WRITE_PGGS writes value to PGGS register
15 IOCTL_READ_PGGS returns PGGS register value
16 IOCTL_ULPI_RESET performs the ULPI reset sequence for resetting the
ULPI transceiver
17 IOCTL_SET_BOOT_HEALTH_STATUS sets healthy bit value to indicate boot health status to
firmware.
18 IOCTL_AFI writes the afi values at given index
Table 60: Description of IOCTLs
IOCTL_
ID
Name
Descript
ion
Arguments
Node_ID Arg1 Arg2 Return Value
0 IOCTL_GET_RPU_OPER_MO
DE
returns
current
RPU
operating
mode
(lockstep/
split)
unused
unused unused Operating
mode
0:
LOCKSTEP
1: SPLIT
1
IOCTL_SET_RPU_OPER_MO
DE
configure
s RPU
operating
mode
(lockstep/
split)
unused
Value of operating
mode
0: LOCKSTEP
1: SPLIT
unused
unused
2 IOCTL_RPU_BOOT_AD
DR_CONFIG
configure
s RPU
boot
address
NODE_RPU
_0
NODE_RPU
_1
Value to set for
boot address
0: LOVEC/TCM
1: HIVEC/OCM
unused
unused
3 IOCTL_TCM_COMB_C
ONFIG
configure
s TCM to
be in split
mode or
combined
mode
unused
Value to set (Split/
Combined)
0: SPLIT
1: COMB
unused
unused
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 208
Table 60: Description of IOCTLs (cont'd)
IOCTL_
ID
Name
Descript
ion
Arguments
Node_ID Arg1 Arg2 Return Value
4 IOCTL_SET_TAPDELAY_BYP
ASS
enables/
disables
tap delay
bypass
unused Type of tap delay
0:
NAND_DQS_IN
1:
NAND_DQS_OU
T
- 2: QSPI
Tap-delay
Enable/ Disable
0: DISABLE
1: ENABLE
unused
5 IOCTL_SET_SGMII_MO DE enables/
disables
SGMII
mode for
the GEM
device
NODE_ETH
_0,
NODE_ETH
_1,
NODE_ETH
_2,
NODE_ETH
_3
"GMII mode
Enable/ Disable
0: DISABLE
1: ENABLE
unused
unused
6 IOCTL_SD_DLL_RESET resets DLL
logic for
the SD
device
NODE_SD_0
,
NODE_SD_1
SD DLL Reset type
0: ASSERT
1: RELEASE
2: PULSE
unused
unused
7 IOCTL_SET_SD_TAPDE LAY sets
input/
output tap
delay for
the SD
device
NODE_SD_0
,
NODE_SD_1
Type of tap delay to
set
0: INPUT
1: OUTPUT
Value to set for
the tap delay
unused
8 IOCTL_SET_PLL_FRAC_
MODE
sets PLL
mode
unused PLL clock ID PLL Mode
0:
FRAC_MOD
E
1:
INT_MODE
unused
9 IOCTL_GET_PLL_FRAC_
MODE
returns
current
PLL mode
unused PLL clock ID unused PLL Mode
0:
FRAC_MOD
E
1:
INT_MODE
10
IOCTL_SET_PLL_FRAC_
DATA
sets PLL
fraction
data
unused PLL clock ID PLL fraction
data
unused
11 IOCTL_GET_PLL_FRAC_
DATA
returns
PLL
fraction
data value
unused PLL clock ID unused PLL fraction
data
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 209
Table 60: Description of IOCTLs (cont'd)
IOCTL_
ID
Name
Descript
ion
Arguments
Node_ID Arg1 Arg2 Return Value
12 IOCTL_WRITE_GGS writes
value to
GGS
register
unused GGS register index
(0/1/2/3)
Register value
to be written
unused
13 IOCTL_READ_GGS returns
GGS
register
value
unused GGS register index
(0/1/2/3)
unused Register value
14 IOCTL_WRITE_PGGS writes
value to
PGGS
register
unused PGGS register
index (0/1/2/3)
Register value
to be written
unused
15 IOCTL_READ_PGGS returns
PGGS
register
value
unused PGGS register
index (0/1/2/3)
unused Register value
16 IOCTL_ULPI_RESET performs
the ULPI
reset
sequence
for
resetting
the ULPI
transceive
r
unused
unused unused unused
17 IOCTL_SET_BOOT_HEA
LTH_STATUS
sets
healthy
bit value
to indicate
boot
health
status to
firmware
unused
healthy bit value unused unused
18 IOCTL_AFI writes the
afi values
at given
index
unused AFI register index
(0 to 15)
Register value
to be written
unused
Query Data
Request data from rmware.
$ echo pm_query_data <query id> <arg1> <arg2> <arg3> > /sys/kernel/debug/
zynqmp-firmware/pm
Enable Clock
Enable the clock for a given clock node_id.
$ echo pm_clock_enable <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 210
Disable Clock
Disable the clock for a given clock node_id.
$ echo pm_clock_disable <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Get Clock State
Get the state of clock for a given clock node_id.
$ echo pm_clock_getstate <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Set Clock Divider
Set the divider value of clock for a given clock node id.
$ echo pm_clock_setdivider <clock id> <divider value> > /sys/kernel/debug/
zynqmp-firmware/pm
Get Clock Divider
Get the divider value of clock for a given clock node_id.
$ echo pm_clock_getdivider <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Set Clock Rate
Set the clock rate for a given clock node_id.
$ echo pm_clock_setrate <clock id> <clock rate> > /sys/kernel/debug/zynqmp-
firmware/pm
Get Clock Rate
Get the clock rate for a given clock node_id.
$ echo pm_clock_getrate <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Set Clock Parent
Set the parent clock for a given clock node_id.
$ echo pm_clock_setparent <clock id> <parent clock id> > /sys/kernel/debug/
zynqmp-firmware/pm
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 211
Get Clock Parent
Get the parent clock for a given clock node id.
$ echo pm_clock_getparent <clock id> > /sys/kernel/debug/zynqmp-firmware/pm
Note: Clock id denions are available in the following le of the clock bindings documentaon:
Documentation/devicetree/bindings/clock/xlnx,zynqmp-clk.txt
PM Platform Driver
The Zynq UltraScale+ MPSoC power management for Linux is encapsulated in a power
management driver, power domain driver and plaorm rmware driver. The system-level API
funcons are exported and as such, can be called by other Linux modules with GPL compable
license. The funcon declaraons are available in the following locaon:
include/linux/firmware/xilinx/zynqmp/firmware.h
The funcon implementaons are available in the following locaon:
drivers/firmware/xilinx/zynqmp/firmware*.c
Provide the correct node in the Linux device tree for proper driver inializaon. The rmware
driver relies on the 'rmware' node to detect the presence of PMU rmware, determine the
calling method (either 'smc' or 'hvc') to the PM-Framework rmware layer and to register the
callback interrupt number.
The ‘rmware’ node contains following properes:
• Compable: Must contain ‘xlnx,zynqmp-rmware’.
• Method: The method of calling the PM framework rmware. It should be ‘smc’.
Note: Addional informaon is available in the following txt le of Linux Documentaon:
Documentation/devicetree/bindings/firmware/xilinx/xlnx,zynqmp-firmware.txt.
Example:
firmware {
zynqmp_firmware: zynqmp-firmware { compatible = "xlnx,zynqmp-firmware";
method = "smc";
};
};
Note: Power domain driver and power management driver binding details are available in the following les
of Linux Documentaon:
•
Documentation/devicetree/bindings/soc/xilinx/xlnx,zynqmp-power.txt
•
Documentation/devicetree/bindings/power/zynqmp-genpd.txt
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 212
Note: xilPM do not support the following EEMI APIs. For current release, they are only supported for Linux
through TF-A.
• query_data
• ioctl
• clock_enable
• clock_disable
• clock_getstate
• clock_setdivider
• clock_getdivider
• clock_setrate
• clock_getrate
• clock_setparent
• clock_getparent
•
pinctrl_request
•
pinctrl_release
•
pinctrl_set_funcon
•
pinctrl_get_funcon
•
pinctrl_set_cong
•
pinctrl_get_cong
Trusted Firmware-A (TF-A)
The Trusted Firmware-A (TF-A) executes in EL3. It supports the EEMI API for managing the
power state of the slave nodes, by sending PM requests through the IPI-based communicaon to
the PMU.
TF-A Application Binary Interface
All APU executable layers below EL3 may indirectly communicate with the PMU via the TF-A.
The TF-A receives all calls made from the lower ELs, consolidates all requests and send the
requests to the PMU.
Following Arm's SMC Calling Convenon, the PM communicaon from the non-secure world to
the TF-A is organized as SiP Service Calls, using a predened SMC funcon idener and SMC
sub-range ownership as specied by the calling convenon.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 213
Note that the EEMI API implementaon for the APU is compliant with the SMC64 calling
convenon only.
EEMI API calls made from the OS or hypervisor soware level pass the 32-bit API ID as the SMC
Funcon Idener, and up to four 32-bit arguments as well. As all PM arguments are 32-bit
values, pairs of two are combined into one 64-bit value.
The TF-A returns up to ve 32-bit return values:
• Return status, either success or error and reason
• Addional informaon from the PM controller
Checking the API Version
Before using the EEMI API to manage the slave nodes, the user must check that EEMI API
version implemented in the TF-A matches the version implemented in the PMU rmware. EEMI
API version is a 32-bit value separated in higher 16 bits of MAJOR and lower 16 bits of MINOR
part. Both elds must be the same between the TF-A and the PMU rmware.
The EEMI version implemented in the TF-A is dened in the local EEMI_API_VERSON ag. The
rich OS may invoke the PM_GET_API_VERSION funcon to retrieve the EEMI API version from
the PMU. If the versions are dierent, this call will report an error.
Note: This EEMI API call is version independent; every EEMI version implements it.
Checking the Chip ID
Linux or other rich OS can invoke the PM_GET_CHIPID funcon via SMC to retrieve the chip ID
informaon from the PMU.
The return values are:
• CSU idcode register (see TRM).
• CSU version register (see TRM).
For more details, see the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Power State Coordination Interface (PSCI)
Power State Coordinaon Interface is a standard interface for controlling the system power state
of Arm processors, such as suspend, shutdown, and reboot. For the PSCI specicaons, see
hp://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0022c/index.html.
TF-A handles the PSCI requests from Linux. TF-A supports PSCI v0.2 only (with no backward
compable support for v0.1).
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 214
The Linux kernel comes with standard support for PSCI. For informaon regarding the binding
between the kernel and the TF-A/PSCI, see hps://www.kernel.org/doc/Documentaon/
devicetree/bindings/arm/psci.txt.
Table 61: PSCI v0.2 Functions Supported by the TF-A
Functions Description Supported
PSCI Version Return the version of PSCI implemented. Yes
CPU Suspend Suspend execution on a core or higher level topology node. This call is
intended for use in idle subsystems where the core is expected to return to
execution through a wakeup event.
Yes
CPU On Power up a core. This call is used to power up cores that either:
• Have not yet been booted into the calling supervisory software.
• Have been previously powered down with a CPU_OFF call.
Yes
CPU Off Power down the calling core. This call is intended for use in hotplug. A core
that is powered down by CPU_OFF can only be powered up again in
response to a CPU_ON.
Yes
Affinity Info Enable the caller to request status of an affinity instance. Yes
Migrate (Optional) This is used to ask a uniprocessor Trusted OS to migrate its context to a
specific core.
Yes
Migrate Info Type
(Optional)
This function allows a caller to identify the level of multicore support
present in the Trusted OS.
Yes
Migrate Info Up CPU
(Optional)
For a uniprocessor Trusted OS, this function returns the current resident
core.
Yes
System Off Shut down the system. Yes
System Reset Reset the system. Yes
PSCI Features Introduced in PSCI v1.0.
Query API that allows discovering whether a specific PSCI function is
implemented and its features.
Yes
CPU Freeze (Optional) Introduced in PSCI v1.0.
Places the core into an IMPLEMENTATION DEFINED low-power state.
Unlike CPU_OFF it is still valid for interrupts to be targeted to the core.
However, the core must remain in the low power state until it a CPU_ON
command is issued for it.
No
CPU Default Suspend
(Optional)
Introduced in PSCI v1.0.
Will place a core into an IMPLEMENTATION DEFINED low-power state.
Unlike CPU_SUSPEND the caller need not specify a power state parameter.
No
Node HW State
(Optional)
Introduced in PSCI v1.0.
This function is intended to return the true HW state of a node in the
power domain topology of the system.
Yes
System Suspend
(Optional)
Introduced in PSCI v1.0.
Used to implement suspend to RAM. The semantics are equivalent to a
CPU_SUSPEND to the deepest low-power state.
Yes
PSCI Set Suspend Mode
(Optional)
Introduced in PSCI v1.0.
This function allows setting the mode used by CPU_SUSPEND to coordinate
power states.
No
PSCI Stat Residency
(Optional)
Introduced in PSCI v1.0.
Returns the amount of time the platform has spent in the given power
state since cold boot.
Yes
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 215
Table 61: PSCI v0.2 Functions Supported by the TF-A (cont'd)
Functions Description Supported
PSCI Stat Count
(Optional)
Introduced in PSCI v1.0.
Return the number of times the platform has used the given power state
since cold boot.
Yes
PMU Firmware
The EEMI service handlers are implemented in the PMU rmware, as one of the modules called
PM Controller (There are other modules running in the PMU rmware to handle other types of
services). For more details, see Chapter 10: Plaorm Management Unit Firmware.
Power Management Events
The PM Controller is event-driven, and all of the operaons are triggered by one of the following
events:
• EEMI API events triggered via IPI0 interrupt.
• Wake events triggered via GPI1 interrupt.
• Sleep events triggered via GPI2 interrupt.
• Timer event triggered via PIT2 interrupt.
EEMI API Events
EEMI API events are soware-generated events. The events are triggered via IPI interrupt when
a PM master iniates an EEMI API call to the PMU. The PM Controller handles the EEMI request
and may send back an acknowledgment (if one is requested.) An EEMI request oen triggers a
change in the power state of a node or a master, with some excepons.
Wake Events
Wake events are hardware-generated events. They are triggered by a peripheral signaling that a
PM master should be woken-up. All wake events are triggered via the GPI1 interrupt.
The following wake events are supported by the PM controller:
• GIC wake events which signal that a CPU shall be woken up due to an interrupt triggered by a
hardware resource to the associated GIC interface. The following GIC wake events are
supported:
○ APU[3:0]An event for each APU processor
○ RPU[1:0]An event for each RPU processor
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 216
• FPD wake event directed by the GIC Proxy. This wake event is triggered when any of the
wake sources enabled prior to suspending. The purpose of this event is to trigger a wake-up of
APU master when FPD is powered down. If FPD is not powered down, none of the wake
signals would propagate through FPD wake. Instead, the wake would propagate through GIC
wake if the associated interrupt at the GIC is properly enabled. All wake events targeted to
the RPU propagate via the associated GIC wake.
Sleep Events
Sleep events are soware-generated events. The events are triggered by a CPU aer it nalizes
the suspend procedure with the aim to signal to the PMU that it is ready to be put in a low power
state. All sleep events are triggered via GPI2 interrupt.
The following sleep events are supported:
• APU[3:0] An event for each APU processor
• RPU[1:0] An event for each RPU processor
When the PM controller PM Controller receives the sleep event for a parcular CPU, the CPU is
put into a low power state.
Timer Event
Timer event is hardware-generated event. It is triggered by a hardware mer when a period of
me expires. The event is used for power management meout accounng and it is triggered via
PIT2 interrupt.
General flow of an EEMI API Call
The following diagram illustrates the sequence diagram of a typical API call, starng with the call
iniated by a PM Master (such as another PU):
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 217
Figure 61: EEMI API Call Sequence Diagram
PM Master PMU Firmware PM Controller PMU ROM
IPI
XEMI API event
handler()
return
acknowledge
return
XEMI API call
X20021-110217
The previous diagram shows four actors, where the rst one represents the PM Master, i.e. either
the RPU, APU, or a MicroBlaze™ processor core. The remaining 3 actors are the dierent
soware layers of the PMU.
First the PMU rmware receives the IPI interrupt. Once the interrupt has been idened as a
power management related interrupt, the IPI arguments are passed to the Power Management
Module. The PM controller then processes the API call. If necessary it may call the PMU ROM in
order to perform power management acons, such as power on or o a power island, or a power
domain.
Chapter 11: Power Management Framework
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 218
Chapter 12
Reset
The Zynq
®
UltraScale+™ MPSoC reset block is responsible for handling both internal and
external reset inputs to the system, and to meet the reset requirements for all the peripherals
and the APU and RPU. The reset block generates resets for the programmable logic part of the
device, and allows independent reset asseron for PS and PL blocks.
This chapter explains the reset mechanisms involved in the system reset and the individual
module resets.
System-Level Reset
The Zynq UltraScale+ MPSoCs let you reset individual blocks such as the APU, RPU, or even
individual power domains like the FPD and LPD. There are mulple, system-level reset opons,
as follows:
• Power-on reset (POR)
• System reset (SRST_B)
• Debug system reset
For more details on the system-level reset ow, see this link to the “Reset System” chapter in the
Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Block-Level Resets
The PS-only reset can be implemented as a subset of system-reset; however, the user must
provide soware that ensures PS-to-PS AXI transacons are gracefully terminated before
iniang a PS-only reset.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 219
PS-Only Reset
The PS-only reset re-boots the PS while that PL remains acve. You can trigger the PS-only reset
by hardware error signal(s) or a soware register write. If the PS-only reset is due to an error
signal, then the error can be indicated to the PL also, so that the PL can prepare for the PR
restart.
The PS-only reset sequence can be implemented as follows:
• [ErrorLogic] Error interrupt is asserted whose acon requires PS-only reset. This request is
sent to PMU as an interrupt.
• [PMU-FW] Set PMU Error (=>PS-only reset) to indicate to PL.
See the PS Only Reset secon in the “Reset System” chapter of the Zynq UltraScale+ Device
Technical Reference Manual (UG1085) describes the PS-only reset sequence.
Note: PS-only reset is not supported in qspi24 mode on systems with a ash size that is greater than 16
MB.
Application Processing Unit Reset
You can independently reset each of the APU CPU core in the soware.
The APU MPCore reset can be triggered by FPD, WDT, or a soware register write; however,
APU MPCore is reset without gracefully terminang requests to and from the APU. The intent is
that you use the LPD in case of catastrophic failures in the FPD. The APU reset is primarily for
soware debug.
The Zynq UltraScale+ Device Technical Reference Manual (UG1085) describes the APU reset
sequence.
APU-Only Reset
APU-only reset is supported in QSPI24, QSPI32, SD0, SD1, SD1-LS, and eMMC boot modes.
Note: The QSPI24 boot mode is not supported by the APU-only restart on systems with ashes greater
than 16 MB in the QSPI single mode and greater than 32 MB in the dual parallel mode.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 220
Real Time Processing Unit Reset
Each Cortex
®
-R5F core can be independently reset. In lockstep mode, only the Cortex-R5F_0
needs to be reset to reset both Cortex-R5F cores. It can be triggered by errors or a soware
register write. The Cortex-R5F reset can be triggered due to a lockstep error to be able to reset
and restart the RPU. It needs to gracefully terminate Cortex-R5F ingress and egress transacons
before iniang reset of corresponding Cortex-R5F.
Full Power Domain Reset
The FPD-reset resets all of the FPD power domain and can be triggered by errors or a soware
register write. If the FPD reset is due to error signal, then the error must be indicated to both the
LPD and the PL.
The FPD reset can be implemented by leveraging the FPD power-up sequence; however, it needs
to gracefully terminate FPD ingress and egress AXI transacons before iniang reset of FPD.
FPD reset sequence can be PL Reset.
The Zynq UltraScale+ MPSoCs has general-purpose output pins from the PMU block that can be
used to reset the blocks in PL. Addionally, GPIO using the EMIO interface can also be used to
reset PL logic blocks. For a detailed descripon of the reset ow, see the this link to the “Reset
System” chapter in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
For more informaon on the soware APIs for reset, see the PMU rmware in Chapter 9:
Plaorm Management.
Warm Restart
The Zynq UltraScale+ MPSoC is a highly complex piece of silicon, capable of running mulple
subsystems on the chip simultaneously. As such, Zynq UltraScale+ supports various types of
reset. This varies from the simplest system reset to the much more complicated subsystem
restart. In any system or subsystem that has a processor component and a programmable logic
component, reset must entail both reset to the hardware as well as soware. Reset to the
hardware includes the following:
• Reseng of the processor and all peripherals associated with the system/subsystem
• Cleaning up of the memory as needed
• Making sure that the interconnect is in a clean state that is capable of roung trac.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 221
Reset to the soware results in the processor starng from the reset vector. However, designer
must make sure that a valid and clean code for the system/subsystem is located at the reset
vector in order to bring the system back to a clean running state.
Resets for Zynq UltraScale+ are broadly divided into two categories. They are:
• Full system resets
• Subsystem restarts
Full system resets include the following:
• Power-On-Reset (POR)
• System-reset
• PS-only-reset
Subsystem restarts include APU subsystems and RPU subsystem restarts.
Full system resets are quite straight forward. Hardware is brought back to the reset state and
soware starts execung ROM code, with a minor behavior dierence between the reset types.
There are subtlees to PS-only reset which will be discussed in later secons.
Subsystem restart is more complicated. A subsystem in Zynq UltraScale+ is composed of all the
components of a parcular operang system. The following gure shows both Vivado's view of
the PS as well as example subsystems as dened by the OS. The default IP conguraon menu in
Vivado provides a aened view, consisng of all available PS components. In the example, these
components are paroned into three separate subsystems, each running an independent
operang system. Each subsystem consists of a processor, list of peripherals and memory. The
example shows the following subsystems:
• RPU based subsystem running uC/OS-II
• RPU based subsystem running FreeRTOS
• APU based subsystem running Linux
Subsystems can be congured in the Isolaon Conguraon view that is inside the Vivado PCW
(PS Conguraon Wizard), when the Advanced Mode check box is enabled.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 222
Figure 62: Vivado IP Configuration Menu
During subsystem restart, the enre subsystem is restarted from a clean state without aecng
the running of the other acve subsystems dened in MPSoC. For example, during an APU
subsystem restart, an APU subsystem running Linux is restarted as far back as FSBL, while the
RPU subsystem running FreeRTOS and uC/OS-II connues to funcon undisturbed. Similarly for
a RPU subsystem restart, an APU subsystem connues to funcon undisturbed.
Subsystem restarts are managed by the plaorm management unit (PMU). To restart each
subsystem, PMU must rst ensure that all on-going AXI-transacons are terminated and that no
new transacons are issued. In the subsystems shown in the following gure, the interconnects
that connects the components of the subsystem, are not explicitly shown. However, each
subsystem includes mulple interconnects and the same interconnects are used by all three
subsystems. If the PMU rmware resets all the components in a subsystem while leaving
unnished transacons in the interconnect, the AXI master and slave might both be in the reset
state. However, the unnished AXI transacons will remain in the interconnect, thus blocking all
subsequent trac. Stuck transacons in the interconnect causes the system to freeze as these
connecons are shared. It is therefore imperave that the PMU ensures all transacons are
completely nished before reseng each and every components in the subsystem, including the
processor.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 223
Figure 63: Subsystem Components for Various Operating Systems
Before releasing the processor from reset, the PMU must ensure that the code in the reset vector
will result in a clean system restart. In the case of the RPU subsystem running standalone
applicaons, this means either loading a clean copy of the applicaon elf or making sure that the
applicaon code is re-entrant. In the case of the APU subsystem running Linux, this means
starng from a re-entrant copy of FSBL.
Note: The on-chip memory (OCM) module contains 256 KB of RAM starng at 0xFFFC0000. The OCM is
mainly used by the FSBL and TF-A components. The FSBL uses the OCM region from 0xFFFC0000 to
0xFFFE9FFF. The last 512B of this region is used by the FSBL to share the hando parameters
corresponding to applicaons that the TF-A hands o. The TF-A uses the rest of the OCM i.e. from
0xFFFEA000 to 0xFFFFFFFF.
The current implementaon of a warm reset requires the FSBL to be in the OCM to support the PMU
rmware hand o to (already exisng) the FSBL without actually restarng. Hence, the OCM is completely
used and no other applicaon is allowed to use it when a warm restart is enabled.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 224
Supported Use Cases
APU Subsystem Restart
For an APU subsystem only restart, you must dene the APU subsystem using PCW in the
Vivado design tools. The PMU executes the funcon to restart the APU subsystem. First, the
PMU idles all components in the APU subsystem. When all is quiet, the PMU will reset each
component, including the APU processors. When the reset is released, it will re-execute the FSBL
code in the OCM. The task carried out by the FSBL for restart diers only slightly than that of
the POR.
Note: The FSBL is re-entrant. Hence, the APU can simply re-execute the FSBL without having to reload a
clean copy.
The following gure shows the APU subsystem restart process.
Figure 64: APU Subsystem Restart Process
The start of this ow diagram represents a clean running state. Linux, RPU, PMU, and CSU
subsystems are in running status. The health of the APU subsystem is monitored by an APU
WDT (watchdog mer). Linux runs a background applicaon which periodically boosts the
watchdog to prevent it from ming out. If an APU subsystem hangs, the WDT mes out. The
meout interrupts the PMU and results in an APU subsystem restart. Alternavely, you can
invoke the APU subsystem restart by directly calling for it in Linux.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 225
Implementation
To support any subsystem restart, a subsystem must rst be dened in the Vivado design tools
using the Isolaon Conguraon view. For an APU subsystem running Linux, the following APU
subsystem are required in addion to the default PMU subsystem:
• A secure APU system for running the FSBL and TF-A
• A non-secure APU subsystem for running Linux.
See Subsystem Power Management for more informaon on subsystem conguraon and an
example of the APU only subsystem.
IMPORTANT! While APU subsystem consists solely of PS components, it is oen the case that APU
subsystem also includes IP peripherals implemented in PL. Unfortunately, isolaon conguraon menu
does not include features to assign PL IPs to dierent subsystems. As a result, all IPs instanated in Vivado
are added to the generated device tree source (DTS) le. In order to properly dene the APU subsystem, all
PL IPs that do not belong in the APU subsystem need to be manually removed from the DTS le.
Otherwise, drivers for all the so IPs will be enabled for Linux, and APU will aempt to manage all the so
IPs even when the APU is going through a warm restart.
IMPORTANT! During a subsystem restart, all components in the subsystem must be in the idle state,
followed by reset. This is implemented for supported components in the PS. For all IPs in PL of a subsystem
that are AXI slaves, no addional tasks are required to idle them. You may supply code to reset these slaves
if desired. For PL IPs that are AXI masters, you must provide the necessary code to stop and complete all
AXI transacons from the master as well as to reset it. See Idle and Reset of Peripherals for details on
adding the idle and reset code.
See GPIO Reset to PL for design issue and guidelines pertaining to using resetn signal from PS
to PL (ps_resetn). You can oponally enable the recovery and escalaon features as desired.
Building Soware for detailed instrucons on building the soware.
RPU Subsystem Restart
RPU as Master
For an RPU subsystem only restart, you must dene the RPU subsystem using PCW in the Xilinx
Vivado
®
Design Suite. The PMU executes the funcon to restart the RPU subsystem. First, the
PMU checks if master is RPU and FSBL was inially running on RPU. Then PMU will idle all
components in the RPU subsystem. When all is quiet, the PMU will reset each component,
including the RPU processors. When the reset is released, it will re-execute the FSBL code. FSBL
for subsystem restart loads only RPU parons without interrupng other subsystems.
Note: RPU only subsystem restart is supported only with FSBL running on RPU just as APU only restart.
Here the FSBL is re-entrant. Hence, the RPU can simply re-execute the FSBL without having to reload a
clean copy.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 226
Once all the subsystems have started and represent a clean running state, the health of the RPU
subsystem can be monitored using an LPD WDT (watchdog mer) by an applicaon running on
RPU. This applicaon must take care of boosng the watchdog to prevent it from ming out. If
an RPU subsystem hangs, this WDT mes out and interrupts the PMU which results in RPU
subsystem restart. For more informaon, see the LPD WDT secon.
Alternavely, you can invoke the RPU subsystem restart by directly calling for it in RPU
applicaon.
Implementation
The implementaon is same as APU only subsystem restart except that RPU subsystem must be
dened in the Vivado
®
Design Suite using the Isolaon Conguraon view.
Note: To support any subsystem restart, a subsystem must rst be dened in the Vivado design tools using
the Isolaon Conguraon view.
The RPU subsystem requires RPU running an FSBL and RPU applicaon in addion to PMU
subsystem. See Subsystem Power Management for more informaon on subsystem conguraon
and an example of the APU only subsystem.
IMPORTANT! During a subsystem restart, all components in the subsystem must be in the idle state,
followed by reset. This is implemented for supported components in the PS. For all IPs in PL of a subsystem
that are AXI slaves, no addional tasks are required to idle them. You may supply code to reset these slaves
if desired. For PL IPs that are AXI masters, you must provide the necessary code to stop and complete all
AXI transacons from the master as well as to reset it. See Idle and Reset of Peripherals for details on
adding the idle and reset code.
See
GPIO Reset to PL for design issue and guidelines pertaining to using resetn signal from PS
to PL (ps_resetn). You can oponally enable the recovery and escalaon features as desired.
See Building Soware for detailed instrucons on building the soware.
APU as Master
RPU subsystem restart requires the APU subsystem and one or more RPU subsystems running in
lock-step or split mode. The APU subsystem running Linux is the master of the RPU subsystems
and manages the life cycle of the subsystem using the remoteproc feature of OpenAMP. APU
uses remoteproc to load, start, and stop the RPU applicaon. It also re-syncs the APU subsystem
with RPU subsystem aer the restart. APU subsystem can trigger a RPU restart by following
sequence:
1. First, it stops the RPU
2. Loads the new rmware
3. Then, it starts the RPU again.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 227
Many events including user command, RPU watchdog meout or message from the RPU to APU
via message pipe may trigger the RPU subsystem restart. Then, APU issues remoteproc
command to PMU to start or stop the RPU, and the PMU changes the state of the RPU
subsystem.
The following gure shows the RPU subsystem restart process.
Figure 65: RPU Subsystem Restart
The start of the above diagram represents a clean running state for all subsystems, Linux, RPU,
PMU and CSU. In the owchart, APU receives a RPU subsystem restart request. When APU
receives the restart request, it uses remoteproc features to stop the RPU subsystem, load new
rmware code, and then starts the RPU subsystem again. The ow chart shows the use of a RPU
WDT. The RPU periodically boosts the watch dog. If the RPU hangs, WDT mes out. Linux will
receive the meout and restarts the RPU subsystem.
Implementation
You must dene the RPU subsystem using the Isolaon Conguraon view in Vivado PCW, and
both PMU and APU subsystems are required. In addion, two conguraons are possible for the
RPU subsystem: RPUs in lock step mode or in split mode. See the Isolaon Conguraon
Consideraon wiki page for more informaon on subsystem conguraon. Sharing of peripherals
between subsystems are not supported. Make sure that the peripherals in all subsystems are
mutually exclusive.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 228
IMPORTANT! In the process of subsystem restart, all components in the subsystem must be in the idle
state, followed by reset. This is implemented for supported components in the PS. For all IPs in PL of a
subsystem that are AXI slaves, no addional tasks are required to idle them. User may supply code to reset
the slaves if desired. For PL IPs that are AXI masters, user must provide the necessary code to stop and
complete all AXI transacons from the master as well as to reset it. See Idle and Reset of Peripherals for
details on adding the idle and reset code.
RPU subsystem restart is supported with Linux kernel implementaon of remoteproc on APU in
conjuncon with OpenAMP library on RPU. It is currently not supported with Linux userspace
OpenAMP library on APU. RPU applicaon must be wrien in accordance with the OpenAMP
applicaon requirements. See Libmetal and OpenAMP for Zynq Devices User Guide (UG1186) for
more informaon. Note that the rpmsg is not required for remoteproc. You can employ rpmsg
feature to provide a communicaon pipe between the two processors. However, remoteproc is
independent of rpmsg. To make remoteproc funcon properly with subsystem restart, RPU
applicaon needs to include a resource table with stac shared memory allocaon. Dynamic
shared memory allocaon is not supported for subsystem restart. You must implement the steps
outlined in How to Write a Simple OpenAMP Applicaon in Libmetal and OpenAMP for Zynq
Devices User Guide (UG1186) to sasfy the remoteproc requirement, but not beyond that. Aer
inializaon, the RPU applicaon needs to signal to the PMU that it is Power Management (PM)
aware by calling XPm_InitFinalize().
Note: If you call XPm_InitFinalize() too early, then the slaves that are not yet inialized are powered
o. They will be powered up again when the RPU applicaon comes around to inialize them, which will
incur some addional power-up latency. See ZU+ Example - PM Hello World wiki page for more
informaon on how to write a PM aware RPU applicaon.
Finally, you must ensure that the address of the reserved memory for RPU code is synchronized
across all layers. It must be dened under memory for both APU and RPU subsystems in the
isolaon conguraon of Vivado. The same address region should be used in the DTS le for
OpenAMP overlay in Linux and again, in resource table and linker script for the RPU applicaon.
See GPIO Reset to PL for design issue and guidelines pertaining to using resetn signal from PS
to PL (ps_resetn). You can oponally enable the recovery and escalaon features as desired.
Building Soware for detailed instrucons on building the soware.
PS-Only Reset
For a PS-only restart, the enre processor system is reset while PL connues to funcon. Prior to
invoking PS-only reset, PMU turns on isolaon between PS and PL, thus clamping the signals
between them in well-dened states. Aer PS-only reset is released, PS executes the standard
boot process starng from the PMU ROM, followed by CSU ROM, then FSBL and so on. During
FSBL, the isolaon between PS and PL is removed.
IMPORTANT!
As the soware has gone through a reset cycle, the state of the hardware IPs in PL which
connue to run during the PS-only reset may become out of sync with the state of the soware which
interfaces or controls the IPs. It is your responsibility to make sure that the soware and hardware states
are properly re-synchronized. In a PS-only reset, you cannot download the bitstream again.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 229
PS-only reset can be iniated by Linux command or watchdog meout or PMU error
management block. If you are interested in PS-only reset without APU/RPU subsystem restart,
subsystem/isolaon conguraon is not required. Linux commands for seng reboot type and
reboot will work without addional modicaons.
System Reset
In a system-reset, the enre hardware, both PS and PL are reset. Aer system reset is released,
PS executes the standard boot process starng from the PMU ROM, followed by CSU ROM,
then FSBL and so on. The following table shows the dierences between system reset and POR:
Table 62: Differences between POR and System Reset
POR System Reset
Reset persistent registers Preserves persistent registers
Resamples boot mode pins Does not resample boot mode pins
Reset debug states Preserves debug states
Resample eFuse values Requires explicit software action to refresh
Security state determined Security state locked
Clear tamper response Preserves tamper response
Select security key source Security key source locked
Optional LBIST and/or SCAN/CLEAR Does not run LBIST or SCAN/CLEAR
Run MBIST Explicit software action needed to run MBIST
System reset can be iniated by Linux command or watchdog meout or PMU error
management block. If you are interested in only System reset without APU/RPU subsystem
restart, subsystem/isolaon conguraon is not required.
Note: System reset is not supported in qspi24 mode on systems with a ash size that is greater than 16
MB.
Idle and Reset of Peripherals
It is necessary to stop/complete any ongoing transacon by any IP or processor of the subsystem
before reseng them. Otherwise, it may lead to hanging of the interconnect and eventually
hanging of the enre system. Also, to ensure proper operaon by the IP aer reboot, it is best to
reset them and bring them to post bootROM state.
PMU rmware implements peripheral idling and reseng for the PS IPs that can be idled / reset
during the subsystem reset. The IPs that will be aempted to idled/reset is based on isolaon
conguraon of the Vivado.
Build PMU rmware with the following idling ags to enable subsystem node idling and reseng:
• ENABLE_NODE_IDLING
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 230
• IDLE_PERIPHERALS
Node Reset and Idle
During a subsystem restart, the PMU rmware makes sure that the associated PS peripheral
nodes are idled and brought to reset state. Following is the list of currently supported PS
peripherals that will undergo idle/reset, if they are part of the subsystem that is undergoing reset:
• TTC
• Ethernet/EMAC
• I2C
• SD
• eMMC
• QSPI
• USB
• DP
• SATA
See GPIO reset to PL to understand the implicaon of GPIO reset.
Note: PS peripherals are idled prior to invoking resets for user invoked reboot of PS-only and system-reset
command.
Custom Hooks
PMU rmware does not keep track of PL peripherals. Hence, there is no idle/reset funcon
implementaon available in the PMU rmware. However, it is necessary to treat those
peripherals in the same the PS peripherals are treated. You can add a custom hook in the
idle_hooks.c le to idle the PL peripherals and reset them. These hooks can be called from
the PmMasterIdleSlaves funcon in the pm_master.c le of the PMU rmware.
lib/sw_apps/zynqmp_pmufw/src/pm_master.c
:dir:dir -769,6 +769,12 :dir:dir static void PmMasterIdleSlaves(PmMaster*
const master)
PmDbg(DEBUG_DETAILED,"%s\r\n", PmStrNode(master->nid));
+ /*
+ * Custom hook to idle PL peripheral before PS peripheral idle
+ */
+
+ Xpfw_PL_Idle_HookBeforeSlaveIdle(master);
+
while (NULL != req) {
u32 usage = PmSlaveGetUsageStatus(req->slave, master); Node = &req->slave-
>node;
:dir:dir -783,6 +789,11 :dir:dir static void PmMasterIdleSlaves(PmMaster*
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 231
const master)
}
req = req->nextSlave;
}
+
+ /*
+ * Custom hook to idle PL peripheral after PS peripheral idle
+ */
+ Xpfw_PL_Idle_HookAfterSlaveIdle(master);
#endif
}
The Xpfw_PL_Idle_HookBeforeSlaveIdle and Xpfw_PL_Idle_HookAfterSlaveIdle
can contain the code to idle the PL peripherals and reset them if necessary. The implementaon
can be either of the following:
• Write AXI registers of PL IPs to bring them to idle state and reset. This is the preferred and a
graceful way to idle PL peripherals.
• Implement a signal based handshake where PMU rmware signals PL to idle all PL IPs. This
implementaon should be used when there is no direct control to gracefully stop trac. For
example, you can use this implementaon if there are non DMA PL IPs, which does not have
reset control but are connected through a rewall IP. This implementaon also allows stopping
all trac passing through it unlike the other where each IP needs to be idled individually.
Note: Implementaon for these custom hooks is not provided by Xilinx.
GPIO Reset to PL
Vivado conguraon allows you to enable fabric resets from PS to PL. The following gure shows
that the Zynq UltraScale+ block outputs pl_resetn0 and pl_resetn1 signals with Fabric
Reset Enabled and the Number of Fabric Resets set to 2, can be used to drive reset pins of PL
components.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 232
Figure 66: Resets from PS to PL
The pl_resetn signals are implemented with PS GPIOs. Pl_resetn pins are released aer
bitstream conguraon in soware using the psu_ps_pl_reset_config_data funcon. In
the case where a subsystem also uses GPIO for purpose other than reset, the GPIO block is
included in the subsystem denion. The image below shows an example of an APU subsystem
with GPIO as a slave peripheral.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 233
Figure 67: APU Subsystem with GPIO
In the case where GPIO is a subsystem slave peripheral, the enre GPIO component will be reset
as part of the restart process when the subsystem is being restarted. Since pl_resetn are
implemented with GPIOs, pl_resetn will be forced low during subsystem restart. This behavior
may be undesirable if the pl_resent signals are being used to drive PL IPs in subsystems other
than the one being reset. For example, if pl_resetn0 drives resets to PL IP for APU subsystem
and pl_resetn1 drives PL IPs for RPU subsystem.
During APU subsystem restart, both pl_resetn0 and pl_resent1 will be forced into the
reset state. Consequently, PL IPs in RPU subsystem will be reset. This is the wrong behavior since
APU-restart should not aect the RPU subsystem as the GPIO is implicitly shared between the
APU and RPU subsystem via pl_resetn signals. Since sharing of peripherals is not supported
for subsystem restart, pl_resetn causes problems during subsystem reset. The work-around is
to skip idling and reseng GPIO peripheral during any subsystem restart even if the component
is assigned in the subsystem/isolaon conguraon.
To skip the GPIO reset during the node Idling and reset, build the PMU rmware with following
ag:
REMOVE_GPIO_FROM_NODE_RESET_INFO
Note: GPIO component goes through a reset cycle also during PS-only reset. PMU rmware enables PS-PL
isolaon prior to calling PS only reset which locks pl_resetn to High. However, as soon as FSBL removes
the PS-PL isolaon, the reset goes Low. FSBL then calls psu_ps_pl_reset_config_data to
recongure pl_resetn back to High. This is needed since reseng the PL components allows proper
synchronizaon of soware and hardware states aer reset.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 234
Recovering from a Hang System
In an event of system hang, as indicated by FPT WDT meout, PMU can be used to carry out a
sequence of events to try and recover from the unresponsive condion. By default, when FPD
WDT mes out, PMU rmware will not invoke any type of restart. This is so that user can specify
the exact desired behavior. However, Xilinx provides a typical recovery scheme in which PMU
rmware monitors the state of APU subsystem using FPD WDT and restart APU (Linux)
subsystem if the mer expires, indicang problem with Linux.
Since RPU subsystem is managed by Linux using remoteproc, the life-cycle of the RPU subsystem
is completely up to Linux. PMU is not involved in deciding when to restart RPU subsystem(s).
RPU hang recovery can also be implemented with help of either soware or hardware watchdog
between APU and RPU subsystems. In that case, the watchdog is congured and handled by
Linux but the heartbeats is provided by RPU applicaon(s). The exact method of deciding when
to restart RPU is up to the user, watchdog is simply one of many possibilies. To enable recovery,
PMU rmware should be built with enabling error management and recovery. Following macros
enable the Recovery feature:
• ENABLE_EM
• ENABLE_RECOVERY
It is also necessary to build TF-A with following ags (see APU Idling for details):
ZYNQMP_WARM_RESTART=1
IMPORTANT!
One TTC mer (mer 9) will be reserved for PMU's use when these compile ags are
enabled.
Watchdog Management
The FPD WDT is used for monitoring APU state. Soware running on APU periodically touch
FPD WDT to keep it from ming out. The occurrence of WDT meout indicates an unexpected
condion on the APU which prevents the soware from running properly and an APU restart is
invoked. FPD WDT is congured by PMU rmware at inializaon stage, but is periodically
serviced by soware running on APU.
The default meout congured for WDT is 60 seconds and can be changed by
RECOVERY_TIMEOUT ag in PMU rmware. When APU subsystem goes into a restart cycle,
FPD WDT is kept running to ensure that the restart lands in a clean running state where soware
running on APU is able to touch the WDT again. Therefore, the meout for the WDT must be
long enough to cover the enre APU subsystem restart cycle to prevent the WDT from ming
out in the middle of restart process. It is advisable to start providing the heartbeat as soon as is
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 235
feasible in Linux. PetaLinux BSP includes recipe to add the watchdog management service in
init.d. As FPD WDT is owned by PMU rmware, it would be unsafe to use full edged Linux
driver for handling WDT. It is advisable to just pump the heartbeats by wring restart key
(0x1999) to restart register (WDT base + 0x8) of the WDT. It can be done through C program
daemon or it can be part of bash script daemon.
It is recommended to be part of idle thread or similar low priority thread, which if hangs you
should consider the subsystem hang.
The following is the snippet of the single heartbeat stroke to the FPD WDT from command
prompt. This can be included in the bash script which runs periodically.
# devmem 0xFD4D0008 32 0x1999
The following wdt-heartbeat applicaon periodically provides the heartbeat to FPD WDT. For
demo purpose this applicaon is launched as daemon. The code from this applicaon can be
implemented in appropriate locaon such as an idle thread of Linux.
#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#define WDT_BASE 0xFD4D0000
#define WDT_RESET_OFFSET 0x8
#define WDT_RESET_KEY 0x1999
#define REG_WRITE(addr, off, val) (*(volatile unsigned int*)(addr
+off)=(val))
#define REG_READ(addr,off) (*(volatile unsigned int*)(addr+off))
void wdt_heartbeat(void)
{
char *virt_addr; int fd;
int map_len = getpagesize();
fd = open("/dev/mem", (O_RDWR | O_SYNC)); virt_addr = mmap(NULL,
map_len, PROT_READ|PROT_WRITE,
MAP_SHARED,
fd, WDT_BASE);
if (virt_addr == MAP_FAILED) perror("mmap failed");
close(fd);
REG_WRITE(virt_addr,WDT_RESET_OFFSET, WDT_RESET_KEY);
munmap((void *)virt_addr, map_len);
}
int main()
{
while(1)
{
wdt_heartbeat(); sleep(2);
}
return 0;
}
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 236
On the expiry of watchdog, PMU rmware receives and handles the WDT interrupt. PMU
rmware idles the subsystem's master CPU i.e., all A53 cores (see APU Idling), and then carries
out APU only restart ow which includes CPU reset and idling and reseng peripherals (see
Peripheral Idling) associated to the subsystem reset.
Note: If ESCALATION is enabled PMU rmware will trigger the appropriate restart ow (which can be
other than APU only restart) as explained in Escalaon secon.
APU Idling
Each A53 is idled by taking them to the WFI state. This is done through Trusted Firmware-A (TF-
A). For idling CPU, the PMU rmware raises TTC interrupt (mer 9) to TF-A, which issues
soware interrupt to each alive A53 core. The respecve cores then clears the pending SGI on
itself and put itself into WFI.
The last core just before going into WFI issues pm_system_shutdown (PMU rmware API) to
PMU rmware, which then performs APU only restart ow.
This feature must be enabled in TF-A for recovery to work properly. It can be enabled by building
TF-A with ZYNQMP_WARM_RESTART=1 ag.
Modifying Recovery Scheme
When ENABLE_RECOVERY is turned on, Xilinx provides a recovery implementaon in which a
FPD WDT meout results in the invocaon of APU subsystem restart. You can easily modify the
recovery behavior by modifying the code. Alternavely, an example of PMU rmware invoking
system-reset on FPD WDT meout is detailed in Xilinx Answer: 69423.
Escalation
If current recovery cannot bring the system back to the working state, the system must escalate
to a more severe type of reset on the next WDT expiry in order to try and recover fully. It is up to
you to decide on the escalaon scheme. A commonly used scheme starts with APU-restart on
the rst watchdog expiraon, followed by PS-only reset on the next watchdog expiraon, then
nally system-reset.
To enable escalaon, PMU rmware must be built with following ags:
ENABLE_ESCALATION
Escalation Scheme
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 237
Default Scheme
Default escalaon scheme checks for the successful pm_system_shutdown call from TF-A for
APU-only restart which happens when the TF-A is able to successfully idle all acve CPUs. If TF-
A is not successful in idling the acve cores, WDT will me out again with the WDT_in_Progess
ag set, resulng in do escalaon.
Escalaon will trigger System level reset. System level reset is dened as PS only reset if PL is
present or System restart if PL is not present.
The following gure shows the ow of the control in case of default escalaon scheme.
Figure 68: Flow of Control for Default Escalation Scheme
Is WDT_In_Progress
Raise Interrupt to
PMU Firmware
No
Do
Escalation
Restart WDT
Set WDT_in_Progress flag
Raise IPI request to ATF for Clearing APU
Sleep and
wait for
event
Restart WDT
Clear WDT_in_Progress flag
Do APU only
restart
Clear all the pending
interrupts on this core
ATF Raises Sw interrupts
for all Active cores
Is Last Active
Core?
Call Pm System shutdown
call for APU only reset
WFI
Each Active core invoked runs same codeIPI to ATF
IPI to PMU-firmware
Legends
AM Trusted Firmware
Hardware
PMU Firmware
Yes
FPD
WDT
Expired
X21016-060618
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 238
Healthy Bit Scheme
Default scheme for escalaon does not guarantee the successful reboot of the system. It only
guarantees the successful role of TF-A to idle the CPU during the recovery. Consider the scenario
in which the FPD_WDT has med out and APU subsystem restart is called in which TF-A is able
to successfully make the pm_system_shutdown call. However, APU subsystem restart is far
from nished aer pm_system_shutdown is called. The restart process can be stuck
elsewhere, such as fsbl, u-boot or Linux init state. If the restart process is stuck in one of the
aforemenoned tasks, FPD_WDT will expire again, causing the same cycle to be repeated as long
as TF-A is loaded and funconing. This cycle can connue indenitely without the system
boong back into a clean running state.
The Healthy Bit scheme solves this problem. In addion to default scheme, the PMU rmware
checks for a Healthy Bit, which is set by Linux on successful boong. On WDT expiry, if Healthy
Bit is set, it indicates that Linux is able to boot into a clean running state, then no escalaon is
needed. However, if Healthy Bit is not set, that means the last restart aempt did not
successfully boot into Linux and escalaon is needed. There is no need to repeat the same type
of restart. PMU rmware will escalate and call a system level reset.
Healthy Bit scheme is implemented using the bit-29 of PMU global general storage register
(PMU_GLOBAL_GLOBAL_GEN_STORAGE0[29]). PMU rmware clears the bit before starng the
recovery or normal reboot and Linux must set this bit to ag a healthy boot.
PMU global registers are accessed through sysfs interface from Linux. Hence, to set the healthy
bit from the Linux, execute the following command (or include in the code):
# echo "0x20000000 0x20000000" > "/sys/devices/platform/firmware/ggs0"
To enable the healthy bit based escalaon scheme, build the PMU rmware with the following
ag:
CHECK_HEALTHY_BOOT
The following gure shows the ow of the control in case of the healthy bit escalaon scheme.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 239
Figure 69: Healthy Bit Escalation Scheme
Is WDT_In_Progress
Healthy Bit Set?
FPD
WDT
Expired
Raise Interrupt to
PMU Firmware
No
Do
Escalation
No
Restart WDT
Yes
Set WDT_in_Progress flag
Raise IPI request to ATF for Clearing APU
Sleep and
wait for
event
Restart WDT
Clear WDT_in_Progress flag
Do APU only
restart
Clear all the pending
interrupts on this core
ATF Raises Sw interrupts
for all Active cores
Is Last Active
Core?
Call Pm System shutdown
call for APU only reset
WFI
Each Active core invoked runs same codeIPI to ATF
IPI to PMU-firmware
Legends
AM Trusted Firmware
Hardware
PMU Firmware
Yes
X21015-060618
Customizing Recovery and Escalation Scheme
By default, when FPD WDT mes out, PMU FW will not invoke any type of restart. While Xilinx
has provided predened RECOVERY and ESCALATION behaviors, users can easily customize
dierent desired schemes.
When FPD _WDT mes out, it calls FpdSwdtHandler. If ENABLE_EM is dened,
FpdSwdtHandler calls XPfw_recoveryHandler. It is otherwise an empty funcon.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 240
In xpfw_mod_em.c,
#ifdef ENABLE_EM
oid FpdSwdtHandler(u8 ErrorId)
{
XPfw_Printf(DEBUG_ERROR,"EM: FPD Watchdog Timer Error (Error ID: %d)\r\n",
ErrorId);
XPfw_RecoveryHandler(ErrorId);
}
#else
void FpdSwdtHandler(u8 ErrorId) { }
Without ENABLE_EM, you can simply update FpdSwdtHandler which will be called at FPD
Timeout. With ENABLE_EM turned on, you need to update XPfw_recoveryHandler.
Similarly, turning on RECOVERY denes the XPfw_RecoveryHandler (see
xpfw_restart.c). Unless RECOVERY is turned on, XPfw_ RecoveryHandler is an empty
funcon and nothing will happen when FPD_WDT mes out.
RecoveryHandler basically follows the ow chart detailed in the Escalaon Scheme secon.
When FPD_WDT mes out, the code follows the progression of orange boxes. If WDT is not
already in progress, Restart WDT, Set WDT_In_Progress ag, Raise TTC (mer 9) interrupt to TF-
A. Then TF-A takes over. It Raises SW interrupt for all acve cores, clear pending interrupts, etc.
(see blue boxes). Essenally, PMU restarts and boosts the WDT, then sends a request to TF-A.
TF-A cleanly idles all four APUs and when they all get to WFI (Last Acve Core is true), TF-A
issues PMU System Shutdown with APU subsystem as argument back to PMU. When PMU gets
this command, it invokes APU subsystem restart.
If ENABLE_ESCALATION is not set, the code never takes the Do Escalaon path. If the
RecoveryHandler hangs for some reason (for example, something went wrong and APU
cannot put all four CPU cores to WFI), it keeps retrying APU restart or hang forever. When
ENABLE_ESCLATION is on and if anything goes wrong during execuon of the owchart, it will
look like WDT is sll in progress (since clear WDT_in_progress ag happens only as the last step),
Do Escalaon will call SYSTEM_RESET instead of trying APU-restart again and again.
To customize recovery and escalaon behavior, use the provided XPfw_recoveryHandler as a
template to provide a customized XPfw_recoveryHandler funcon.
Building Software
All the soware components are built and packaged by Xilinx PetaLinux tool. See PetaLinux wiki
page for more informaon on how to build and package soware components.
Build Flag for Restart Solution
Following build me ags are not set by default and can alter the behavior of the restart in
Zynq UltraScale+ MPSoC:
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 241
Table 63: Build Time Flags
Component Flag Name Description
PMU firmware
ENABLE_EM
Enable error management and
provide WDT interrupt handling.
This is not directly related to
restart solution but needed for
recovery.
ENABLE_RECOVERY
Enable Recovery during WDT
expiry
ENABLE_ESCALATION Allow escalation on failure of boot
or recovery
CHECK_HEALTHY_BOOT Use Healthy bit to determine
escalation
IDLE_PERIPHERALS ENABLE_NODE_IDLING Both the flags must be used
together to allow PMU firmware to
attempt peripherals node idling
(and reset).
REMOVE_GPIO_FROM_NODE_RESET_INFO Skips GPIO from the node idling
and resetting list.
This is needed when the system is
using GPIO to provide reset (or
similar) signals to PL or other
peripherals outside current
subsystem.
If this flag is set, GPIO is not reset.
TF-A
ZYNQMP_WARM_RESTART=1 Enable WARM RESTART recovery
feature in TF-A that allow the CPU
idling triggered from PMU
firmware.
FSBL FSBL_PROT_BYPASS Skip XMPU/XPPU based
configuration for system except
for DDR and OCM.
Linux CONFIG_SRAM Needed for Remoteproc to work
for load RPU images in the TCM.
Modifying Component Recipes
Each component's recipe can be changed to either include the build me compilaon ags or to
include patches for custom code modicaon/addion. PetaLinux provides meta-user Yocto
based layer for user specic modicaons. The layer can be found in project directory project-
spec/meta-user/ locaon.
PMU Firmware
User specic recipe for PMU rmware can be found in the following locaon:
dir:project-spec/meta-user/recipes-bsp/pmu/pmu-firmware_%.bbappend (if
doesn't exist please create this le at this path).
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 242
The PMU rmware code can be modied by patches against embeddedsw GitHub repo. Locaon
for the source code is embeddedsw/tree/master/lib/sw_apps/zynqmp_pmufw. The
patches should be copied to project-spec/meta-user/recipes-bsp/pmu/files
directory and the same patch names should be added pmu-firmware_%.bbappend le.
Example:
If my_changes.patch (against PMU rmware source) is to be added and all the ags explained
in the Build Time Flags in Building Soware are to be enabled (set), then project-spec/
meta-user/recipes-bsp/pmu/pmu-firmware_%.bbappend may look like the following
le:
YAML_COMPILER_FLAGS_append = " -O2 -DENABLE_EM -DENABLE_RECOVERY
-DENABLE_ESCALATION -DENABLE_NODE_IDLING -DREMOVE_GPIO_FROM_NODE_RESET_INFO
-DCHECK_HEALTHY_BOOT -DIDLE_PERIPHERALS"
FILESEXTRAPATHS_prepend := "${THISDIR}/files:" SRC_URI_append = " file://
my_changes.patch"
FSBL
User specic recipe for the FSBL can be found in the following locaon:
dir:project-spec/meta-user/recipes-bsp/fsbl/fsbl_%.bbappend (if does not
exist, please create this le at this path). The FSBL code can be modied by patches against
embeddedsw GitHub repo. Locaon for the source code is as follows:
embeddedsw/tree/master/lib/sw_apps/zynqmp_fsbl
The patches should be copied to project-spec/meta-user/recipes-bsp/fsbl/files
directory and the same patch names should be added to fsbl_%.bbappend le.
Example:
If my_changes.patch (against the FSBL source) is to be added and all the ags explained in the
Build Time Flags in Building Soware are to be enabled (set), then the modied project-
spec/meta-user/recipes-bsp/fsbl/fsbl_%.bbappend le will look like the following
le (XPS_BOARD_ZCU102 ag was already exisng):
YAML_COMPILER_FLAGS_append = " -DXPS_BOARD_ZCU102 -DFSBL_PROT_BYPASS"
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI_append = " file://my_changes.patch"
TF-A
User specic recipe for TF-A can be found in the following locaon:
dirproject-spec/meta-user/recipes-bsp/arm-trusted-firmware/arm-
trusted-firmware_%.bbappend le (if it doesn't exist, create this le in this path). You can
nd the ATF les in Git repository for arm trusted rmware.
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 243
Example:
To add warm restart ag to TF-A, project-spec/meta-user/recipes-bsp/arm-
trusted-firmware/arm-trusted-firmware_%.bbappend will look like the following le:
#
# Enabling warm restart feature
#
EXTRA_OEMAKE_append = " ZYNQMP_WARM_RESTART=1"
Linux
There are many ways to add /modify Linux conguraon. See PetaLinux Tools Documentaon:
Reference Guide (UG1144) for the same.
User specic recipe for Linux kernel can be found in the following locaon:
project-spec/meta-user/recipes-kernel/linux/linux-xlnx_%.bbappend (if it
doesn't exist, create this le at this path).
You can nd the Linux les at Git Repository for Linux Example:
To add SRAM cong to Linux, create the following bsp.cfg le:
project-spec/meta-user/recipes-kernel/linux/linux-xlnx/bsp.cfg
CONFIG_SRAM=y
Add this le in the following bbapend le of Linux:
project-spec/meta-user/recipes-kernel/linux/linux-xlnx_%.bbappend
SRC_URI += "file://bsp.cfg"
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
Modifying Device Tree
User specic recipe for device tree can be found in the following locaon:
project-spec/meta-user/recipes-bsp/device-tree/device-tree-generation_
%.bbappend. This le contains the following contents:
SRC_URI_append ="\ file://system-user.dtsi \
"
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 244
The content of system-user.dtsi in project-spec/meta-user/recipes-bsp/
device-tree/files directory is as follows:
/include/ "system-conf.dtsi"
/ {
};
This le can be modied to extend the device tree funconality by adding, removing, or
modifying the DTS nodes.
Example: Adding DT node(s) [ remoteproc RPU split mode]
The overlay dtsi(s) can be added in les/ directory (remember to update bbappend le
accordingly) and included in system-user.dtsi. For adding remoteproc related entries to
enable RPU subsystem to load, unload, or restart, add a new overlay le called
remoteproc.dtsi.
Note: This is for split mode. Check open amp documentaon for lockstep and other possible
conguraons.
File: remoteproc.dtsi
/ {
reserved-memory {
#address-cells = <2>;
#size-cells = <2>; ranges;
rproc_0_reserved: rproc:dir3ed000000 { no-map;
reg = <0x0 0x3ed00000 0x0 0x1000000>;
};
};
power-domains {
pd_r5_0: pd_r5_0 {
#power-domain-cells = <0x0>; pd-id = <0x7>;
};
pd_r5_1: pd_r5_1 {
#power-domain-cells = <0x0>; pd-id = <0x8>;
};
pd_tcm_0_a: pd_tcm_0_a {
#power-domain-cells = <0x0>; pd-id = <0xf>;
};
pd_tcm_0_b: pd_tcm_0_b {
#power-domain-cells = <0x0>; pd-id = <0x10>;
};
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 245
pd_tcm_1_a: pd_tcm_1_a {
#power-domain-cells = <0x0>;
pd-id = <0x11>;
};
pd_tcm_1_b: pd_tcm_1_b {
#power-domain-cells = <0x0>; pd-id = <0x12>;
};
};
amba {
r5_0_tcm_a: tcm:dirffe00000 { compatible = "mmio-sram";
reg = <0x0 0xFFE00000 0x0 0x10000>;
pd-handle = <&pd_tcm_0_a>;
};
r5_0_tcm_b: tcm:dirffe20000 { compatible = "mmio-sram";
reg = <0x0 0xFFE20000 0x0 0x10000>;
pd-handle = <&pd_tcm_0_b>;
};
r5_1_tcm_a: tcm:dirffe90000 { compatible = "mmio-sram";
reg = <0x0 0xFFE90000 0x0 0x10000>;
pd-handle = <&pd_tcm_1_a>;
};
r5_1_tcm_b: tcm:dirffeb0000 { compatible = "mmio-sram";
reg = <0x0 0xFFEB0000 0x0 0x10000>;
pd-handle = <&pd_tcm_1_b>;
};
elf_ddr_0: ddr:dir3ed00000 { compatible = "mmio-sram";
reg = <0x0 0x3ed00000 0x0 0x40000>;
};
elf_ddr_1: ddr:dir3ed40000 { compatible = "mmio-sram";
reg = <0x0 0x3ed40000 0x0 0x40000>;
};
test_r50: zynqmp_r5_rproc:dir0 {
compatible = "xlnx,zynqmp-r5-remoteproc-1.0";
reg = <0x0 0xff9a0100 0x0 0x100>, <0x0 0xff340000 0x0 0x100>, <0x0
0xff9a0000 0x0 0x100>;
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 246
reg-names = "rpu_base", "ipi", "rpu_glbl_base"; dma-ranges;
core_conf = "split0"; sram_0 = <&r5_0_tcm_a>; sram_1 = <&r5_0_tcm_b>;
sram_2 = <&elf_ddr_0>; pd-handle = <&pd_r5_0>;
interrupt-parent = <&gic>; interrupts = <0 29 4>;
} ;
test_r51: zynqmp_r5_rproc:dir1 {
compatible = "xlnx,zynqmp-r5-remoteproc-1.0";
reg =<0x0 0xff9a0200 0x0 0x100>, <0x0 0xff340000 0x0 0x100>, <0x0
0xff9a0000 0x0 0x100>;
reg-names = "rpu_base", "ipi", "rpu_glbl_base"; dma-ranges;
core_conf = "split1"; sram_0 = <&r5_1_tcm_a>; sram_1 = <&r5_1_tcm_b>;
sram_2 = <&elf_ddr_1>; pd-handle = <&pd_r5_1>;
interrupt-parent = <&gic>; interrupts = <0 29 4>;
} ;
};
};
Now include this node in system-user.dtsi:
/include/ "system-conf.dtsi"
/include/ "remoteproc.dtsi"
/ {
};
For informaon on OpenAMP and remoteproc, see the OpenAmp wiki page.
Example: Removing DT node(s) [ PL node]
It is necessary to remove PL nodes, which are not accessed or dependent on APU subsystem,
from the device tree. Again, you can modify system-user.dtsi in project-spec/meta-
user/recipes-bsp/device-tree/files to remove specic node or property.
For example, you can modify the system-user.dtsi as following, if you are willing to remove
AXI DMA node from the dts:
/include/ "system-conf.dtsi"
/include/ "remoteproc.dtsi"
/ {
/delete-node/axi-dma;
};
Chapter 12: Reset
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 247
Chapter 13
High-Speed Bus Interfaces
The Zynq
®
UltraScale+™ MPSoC has a serial input/output unit (SIOU) for a high-speed serial
interface. It supports protocols such as PCIe
®
, USD 3.0, DisplayPort, SATA, and Ethernet
protocols.
• The SIOU block is part of the full-power domain (FPD) in the PS.
• The USB and Ethernet controller blocks that are part of the low-power domain (LPD) in the
Zynq UltraScale+ MPSoC also share the PS-GTR transceivers.
• The interconnect matrix enables mulplexing of four PS-GTR transceivers in various
combinaons across mulple controller blocks.
• A register block controls or monitors signals within the SIOU.
This chapter explains the conguraon ow of the high-speed interface protocols.
See this link to the “High-Speed PS-GTR Transceiver Interface” of the Zynq UltraScale+ Device
Technical Reference Manual (UG1085) for more informaon.
USB 3.0
The Zynq UltraScale+ MPSoC USB 3.0 controller consists of two independent dual-role device
(DRD) controllers. Both can be individually congured to work as host or device at any given
me. The USB 3.0 DRD controller provides an eXtensible host controller interface (xHCI) to the
system soware through the advanced eXtensible interface (AXI) slave interface.
• An internal DMA engine is present in the controller and it uses the AXI master interface to
transfer data.
• The three dual-port RAM conguraons implement the RX data FIFO, TX data FIFO, and the
descriptor/register cache.
The following ow diagrams illustrate how to congure USB as mass storage device.
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 248
Figure 70: USB Example Flow: USB Initialization
USB config initialize
Start
Call a function to hook up the handler for control packets
Is Req type==
Std dev req?
Stall on endpoint 0
Call a function to hook up the handler for mass storage
Stall the endpoint 0
Handle standard device request
Is Req type==
class req?
Handle the class request
Is Req type==
vendor req?
Do nothing
Is Req ==
Mass storage
reset?
Do nothing
(Status phase is handled by driver)
Is Req ==
Get_Maz_LUN
Prepare a URB with number of LUNs
Yes
Yes
Yes
Yes
Yes
No
No
No
No
No
A
X15463-111020
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 249
Figure 71: Example USB Flow: Hookup Bulk in and Bulk out Handlers and Initialize
Interrupt Controller
Prepare device descriptors
Prepare configuration descriptors
Call a function to hook up the bulkout handler
Is phase ==
Data?
Send command status wrapper (CSW)
Call a function to hook up the bulk In handler
Read the packet data into Receive Buffer
Is phase ==
Status?
Initialize the driver interrupt controller
Config initialize for GIC
Connect the interrupt controller
Is phase ==
Command?
Parse CBW
Send reduced block command (RBC)
Write operation
Is phase ==
Data?
Is
RBC
Mode sense ==
1?
Send CSW with mode
Yes
Yes
No
No
Yes
No
Yes
Yes
Send CSW with mode
No
No
B
A
X15477-071017
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 250
Figure 72: Enable Interrupts and Start the USB Controller
Enable all the required interrupts
Connect the interrupt controller to the interrupt
handling logic in ARM
Enable interrupts in the ARM
Wait for interrupts
Start USB Controller
B
X15478-021317
For more informaon on USB controller, see this link to the “USB 2.0/3.0 Host, Device, and
Controller,” chapter of the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Gigabit Ethernet Controller
The gigabit Ethernet controller (GEM) implements a 10/100/1000 Mb/s Ethernet MAC
compable with IEEE Standard for Ethernet (IEEE Std 802.3-2008) and is capable of operang in
either half or full-duplex mode in 10/100 mode and full-duplex in 1000 mode.
The processor system (PS) is equipped with four gigabit Ethernet controllers. Registers are used
to congure the features of the MAC, and select dierent modes of operaon. The DMA
controller connects to memory through the advanced eXtensible interface (AXI). It is aached to
the FIFO interface of the controller of the MAC to provide a scaer-gather type capability for
packet data storage in an embedded processing system.
The following gures illustrate an example for conguring an Ethernet controller to send a single
packet of data in RGMII mode.
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 251
Figure 73: Example Ethernet Flow: Initialize Ethernet Controller
Start
Setup UART
Get the Configuration of Ethernet
Hardware
Get Cache Coherence Selection
Initialize Ethernet hardware, setup interrupts and
callbacks
Error in initializing?
Return error and exit
Set the MAC address
Set the loopback speed to 1G
Get the PHY interface
Detect the PHY address
Read the PHY Model
Yes
No
A
X15462-071017
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 252
Figure 74: Example Ethernet Flow: Configure the Ethernet Parameters & Initiate the
Transmit
Clear the PHY of any existing bits
RGMII mode PHY specific register initialization
Configure the Interface modes
Set the speed and put the PHY in reset
Put the PHY in loopback
Return Error and Exit
Error Setting the
PHY loopback?
Set PHY <-> MAC Data clock
Delay
Setup BD space
Setup attributes of BD space
Set up the packet to be transmitted
Clear out the receive packet memory area
Calculate the frame length (not including FCS)
Setup BD Rings and push the Frame
Start the Ethernet Device and Initiate Transmit
Wait for status of the transmitted packet
No
Yes
A
B
X15479-071017
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 253
Figure 75: Example Ethernet Flow: Receive and Validate the Data
Wait for status of the transmitted packet
Receive the packet
Return error and exit
Error
Get the length of the arrived data
Read the packet data received
Verify the received frame length
Validate the frame data
Stop Ethernet Hardware and disable
Interrupts
No
Yes
Stop
B
X15480-021317
For more informaon on Ethernet Controller, see this link to the “Gigabit Ethernet Controller”
chapter in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
PCI Express
The Zynq UltraScale+ MPSoC provides a controller for the integrated block for PCI™ Express
v2.1 compliant, AXI-PCIe Bridge, and DMA modules. The AXI-PCIe Bridge provides high-
performance bridging between PCIe and AXI.
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 254
The following ow diagrams illustrate an example for conguring PCIe root complex for a data
transfer.
Figure 76: Example PCIe Flow: Enable the Legacy Interrupts and Create PCIe Root Bus
Map the register memory space for PCIe bridge
and controller
Start
Map the memory space for ECAM
Write the bridge offset in the bridge register base
Enable the bridge in the bridge control register
Disable the DMA channel registers
Enable the bridge configuration interrupt
Enable Ingress Subtractive decode translation
Enable message filtering
Get the PCIe link up
ECAM
Bit==1
?
Return error and exit
Enable ECAM in ECAM control register
Yes
No
A
X15481-071217
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 255
Figure 77: Example PCIe Flow: Enable the Legacy Interrupts and Create PCIe Root Bus
Get the legacy interrupt number
Return error and exit
Invalid interrupt
number
Register the legacy interrupt handler
Return error and exit
Failed to
register handle?
Enable all legacy interrupts
Get the bridge resources
Error getting
resources?
Create PCIe root bus
Return error and exit
Return error and exit
Error creating
root bus?
Yes
Yes
Yes
Yes
No
No
No
No
A
B
X15483-043021
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 256
Figure 78: Example PCIe Flow: Enable MSI Interrupts and Wait for Interrupts
Assign MSI chip hooks
No
Create the IRQ domain
Enable MSII and MSII status
Error creating
IRQ domain?
Return error and exit
Yes
No
MSII bit Present?
No
Write the MSII low and high addresses
Disable and clear all high range MSI interrupts
Get the MSI_1 IRQ number
Disable and clear all low range MSI interrupts
Get the MSI_0 IRQ number
Enable all low range interrupts
Enable all high range interrupts
Remove the Interrupt Domain
Error Registering
MSI_0 handle?
No
Yes
Register the MSI_0 handle
Error getting the IRQ number?
No
Error Registering
MSI_1 handle?
Yes
Yes
Register the MSI_1 handle
Error getting the IRQ number?
Yes
Yes
No
No
Is MSI bit set?
Yes
B
Scan the PCIe child bus
Assign unassigned bus resoources
Add PCIe bus devices
Set the platform driver data
Wait for interrupts
X15484-043021X15484-071217
Note: For endpoint operaon, refer to this link to “Controller for PCI Express” in the Zynq UltraScale+ Device
Technical Reference Manual (UG1085).
Aer the memory space for PCIe bridge and ECAM is mapped, ECAM is enabled for ECAM
translaons. You then acquire the bus range to set up the bus numbers, and write the primary,
secondary, and subordinate bus numbers. The interrupt system must be set up by enabling all the
miscellaneous and legacy interrupts. You can parse the ranges property of a PCI host bridge
device node, and setup the resource mapping based on its content.
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 257
To create a root bus, allocate the PCIe root bus and add inial resources to the bus. If the MSI bit
is set, you must enable the message signaling interrupt (MSI). Aer conguring the MSI
interrupts, scan the PCIe slot and enumerate the enre PCIe bus and allocate bus resources to
scanned buses. Now, you can add PCIe devices to the system.
For more informaon on PCI Express, see this link to the “DMA Controller” secon and this link
to “Controller for PCI Express” in the Zynq UltraScale+ Device Technical Reference Manual
(UG1085).
Chapter 13: High-Speed Bus Interfaces
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 258
Chapter 14
Clock and Frequency Management
The Zynq
®
UltraScale+™ MPSoC architecture includes a programmable clock generator that
takes a clock of a denite input frequency and generates mulple-derived clocks using the
phase-locked loop (PLL) blocks in the PS. The output clock from each of the PLLs is used as a
reference clock for the dierent PS peripherals.
Unlike the USB and Ethernet peripherals, some peripherals like the UART and SD allow you to
dynamically change the device frequency seng.
This chapter provides informaon about changing the operang frequency of these peripherals
dynamically. See Chapter 11: Power Management Framework for more informaon on reducing
or adjusng the clock frequencies.
Changing the Peripheral Frequency
You can change the peripheral operaon frequency by directly seng the frequency in the
corresponding peripheral clock conguraon register. The Zynq UltraScale+ MPSoC BSP
provides APIs that aid in changing the peripheral clock frequency dynamically according to your
requirements.
The following table shows the standalone APIs that can be used to change the frequency of the
peripherals
Table 64: Standalone APIs
APIs Description
XSDPS_change_clkfreq Change the clock frequency of SD.
XSPIPS_setclkprescaler XSPIPS_getclkprescaler Pre-scale the SPI frequency.
XRtcPSu_calculatecalibration Change the oscillator frequency.
XQSPIPSU_setclkprescaler Change the clock frequency of QSPI.
Chapter 14: Clock and Frequency Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 259
In case of a Linux applicaon, the frequency of all the peripherals is set in the device tree le. The
following code snippet shows the seng of peripheral clock.
ps7_qspi_0: ps7-qspi:dir0xFF0F0000 {
#address-cells = <0x1>;
#size-cells = <0x0>;
#bus-cells = <0x1>;
clock-names = “ref_clk”, “pclk”;
compatible = “xlnx,usmp-gqspi”, “cdns,spi-r1p6”; stream-connected-dma =
<0x26>;
clocks = <0x1e 0x1e>; dma = <0xb>; interrupts = <0xf>;
num-chip-select = <0x2>;
reg = <0x0 0xff0f0000 0x1000 0x0 0xc0000000 0x8000000>;
speed-hz = <0xbebc200>; xlnx,fb-clk = <0x1>;
xlnx,qspi-clk-freq-hz = <0xbebc200>; xlnx,qspi-mode = <0x2>;
To avoid any error condion, the peripheral needs to be stopped before changing the
corresponding clock frequency.
The steps to follow before changing the clock frequency for any peripheral are as follows:
1. Stop the transion pertaining to the peripheral (IP) and make it idle.
2. Stop the IP by appropriately conguring the registers.
3. Change the clock frequency of the peripheral.
4. Issue so reset to the IP.
5. Restart the IP.
For more informaon on Zynq UltraScale+ MPSoC clock generator, see this link in the “Clocking”
chapter in the in the Zynq UltraScale+ Device Technical Reference Manual (UG1085).
Chapter 14: Clock and Frequency Management
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 260
Chapter 15
Target Development Platforms
This chapter describes various development plaorms available for the
Zynq
®
UltraScale+™ MPSoC, such as Quick Emulators (QEMU) and the Zynq UltraScale+ MPSoC
boards and kits.
QEMU
QEMU is a system emulaon model that funcons on an Intel-compable Linux host system.
Xilinx
®
QEMU implements a framework for generang custom machine models based on a
device tree passed to it using the command line. See the Xilinx Quick Emulator User Guide: QEMU
for more informaon about QEMU.
Boards and Kits
Xilinx provides the Zynq UltraScale+ MPSoC ZCU102, ZCU106, and ZCU111 Evaluaon Kits for
developers. To understand more about the evaluaon kits, see the following documentaon
pages:
• ZCU102
• ZCU106
• ZCU111
To know the dierent Zynq UltraScale+ MPSoCs, see the Zynq UltraScale+ MPSoC Products
Page.
Chapter 15: Target Development Platforms
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 261
Chapter 16
Boot Image Creation
Zynq
®
UltraScale+™ MPSoC supports both secure and non-secure boong. While deploying the
devices in eld, it is important to prevent unauthorized or modied code from being run on these
devices. Zynq UltraScale+ MPSoC provides the required condenality, integrity, and
authencaon to host applicaons securely. For more informaon on security features, see Zynq
UltraScale+ Device Technical Reference Manual (UG1085).
Zynq UltraScale+ MPSoCs typically have many hardware and soware binaries that are used to
boot them to funcon as designed and expected. These binaries includes FPGA bitstreams,
Firmware, boot loaders, operang system, and applicaons that you select. For example: FPGA
bitstream les, rst stage boot loader (FSBL), PMU rmware, TF-A, U-Boot, Linux kernel, Roos,
device tree, standalone or RTOS applicaons and so on). Xilinx provides a standalone tool,
Bootgen, to stch all these binary images together and generate a device bootable image in a
specic format that Xilinx loader programs can interpret while loading.
Bootgen has mulple aributes and commands that dene its behavior while generang boot
images. They are secure boot image generaon, non-secure boot image generaon, Secure key
generaon, HMI Mode and so on. For complete details of how to get the Bootgen tool, the
installaon procedure, and details of Zynq Ultrascale+ Boot Image format, Bootgen commands,
aributes, and boot image generaon procedure with examples, see Bootgen User Guide
(UG1283).
Chapter 16: Boot Image Creation
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 262
Appendix A
Libraries
See OS and Libraries Document Collecon (UG643) for informaon on API reference for the
following libraries.
• Standalone Library
• LwIP 2.1.1 Library
• XilIFS
• XilFFS
• XilSecure
• XilSkey
• XilPM
• XilFPGA
• XilMailbox
Appendix A: Libraries
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 263
Appendix B
Additional Resources and Legal
Notices
Xilinx Resources
For support resources such as Answers, Documentaon, Downloads, and Forums, see Xilinx
Support.
Documentation Navigator and Design Hubs
Xilinx
®
Documentaon Navigator (DocNav) provides access to Xilinx documents, videos, and
support resources, which you can lter and search to nd informaon. To open DocNav:
• From the Vivado
®
IDE, select Help → Documentaon and Tutorials.
• On Windows, select Start → All Programs → Xilinx Design Tools → DocNav.
• At the Linux command prompt, enter docnav.
Xilinx Design Hubs provide links to documentaon organized by design tasks and other topics,
which you can use to learn key concepts and address frequently asked quesons. To access the
Design Hubs:
• In DocNav, click the Design Hubs View tab.
• On the Xilinx website, see the Design Hubs page.
Note: For more informaon on DocNav, see the Documentaon Navigator page on the Xilinx website.
References
Appendix B: Additional Resources and Legal Notices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 264
Xilinx References
1. Xilinx Third-Party Licensing Soluon Center
2. PetaLinux Product Page
3. Xilinx Vivado Design Suite – HLx Edions
4. Xilinx Third-Party Tools
5. Zynq UltraScale+ MPSoC Product Table
6. Zynq UltraScale+ MPSoC Product Advantages
7. Zynq UltraScale+ MPSoC Products Page
Zynq Devices Documentation
1. Xilinx Quick Emulator User Guide: QEMU
2. UltraScale Architecture and Product Data Sheet: Overview (DS890)
3. Isolaon Methods in Zynq UltraScale+ MPSoCs (XAPP1320)
4. Zynq UltraScale+ Device Technical Reference Manual (UG1085)
5. Zynq UltraScale+ Device Register Reference (UG1087)
6. Zynq UltraScale+ MPSoC: Embedded Design Tutorial (UG1209)
7. Zynq UltraScale+ MPSoC Processing System LogiCORE IP Product Guide (PG201)
8. UltraScale Architecture System Monitor User Guide (UG580)
9. Libmetal and OpenAMP for Zynq Devices User Guide (UG1186)
10. Embedded Energy Management Interface Specicaon (UG1200)
11. UltraFast Embedded Design Methodology Guide (UG1046)
12. Zynq-7000 SoC: Embedded Design Tutorial (UG1165)
13. Zynq-7000 SoC Soware Developers Guide (UG821)
14. UltraScale Architecture PCB Design User Guide (UG583)
15. Vivado Design Suite Documentaon
16. Bootgen User Guide (UG1283)
Vitis Software Platform and PetaLinux Documents
1. Vis Unied Soware Plaorm Documentaon
Appendix B: Additional Resources and Legal Notices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 265
2. OS and Libraries Document Collecon (UG643)
3. Embedded Design Tools Download
4. PetaLinux Tools Documentaon: Reference Guide (UG1144)
5. Xilinx Soware Development Kit: System Performance (UG1145)
Xilinx IP Documents
1. AXI Central Direct Memory Access LogiCORE IP Product Guide (PG034)
2. AXI Video Direct Memory Access LogiCORE IP Product Guide (PG020)
Miscellaneous Links
1. Xilinx Github
2. Embedded Development
3. Yocto
4. PetaLinux Soware Development
5. Zynq UltraScale+ Silicon Devices Page
6. Xilinx Answer: 66249
7. Vivado Quick Take Video: Vivado PS Conguraon Wizard Overview
8. Xilinx Wiki
9. Kria SOM
10. WP512
11. WP516
Third-Party References
1. Lauterbach Technologies
2. Arm Trusted Firmware
3. Xen Hypervisor
4. Arm Developer Center
5. Arm Cortex-A53 MPCore Processor Technical Reference Manual
6. Yocto Product Development
7. GNU FTP
Appendix B: Additional Resources and Legal Notices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 266
8. Power State Coordinaon Interface – Arm DEN 0022B.b, 6/25/2013
Revision History
The following table shows the revision history for this document.
Section
Revision Summary
11/02/2022 Version 2022.2
General updates Updated a few technical details
Please Read: Important Legal Notices
The informaon disclosed to you hereunder (the "Materials") is provided solely for the selecon
and use of Xilinx products. To the maximum extent permied by applicable law: (1) Materials are
made available "AS IS" and with all faults, Xilinx hereby DISCLAIMS ALL WARRANTIES AND
CONDITIONS, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING BUT NOT LIMITED TO
WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, OR FITNESS FOR ANY
PARTICULAR PURPOSE; and (2) Xilinx shall not be liable (whether in contract or tort, including
negligence, or under any other theory of liability) for any loss or damage of any kind or nature
related to, arising under, or in connecon with, the Materials (including your use of the
Materials), including for any direct, indirect, special, incidental, or consequenal loss or damage
(including loss of data, prots, goodwill, or any type of loss or damage suered as a result of any
acon brought by a third party) even if such damage or loss was reasonably foreseeable or Xilinx
had been advised of the possibility of the same. Xilinx assumes no obligaon to correct any
errors contained in the Materials or to nofy you of updates to the Materials or to product
specicaons. You may not reproduce, modify, distribute, or publicly display the Materials
without prior wrien consent. Certain products are subject to the terms and condions of
Xilinx's limited warranty, please refer to Xilinx's Terms of Sale which can be viewed at hps://
www.xilinx.com/legal.htm#tos; IP cores may be subject to warranty and support terms contained
in a license issued to you by Xilinx. Xilinx products are not designed or intended to be fail-safe or
for use in any applicaon requiring fail-safe performance; you assume sole risk and liability for
use of Xilinx products in such crical applicaons, please refer to Xilinx's Terms of Sale which can
be viewed at hps://www.xilinx.com/legal.htm#tos.
Appendix B: Additional Resources and Legal Notices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 267
AUTOMOTIVE APPLICATIONS DISCLAIMER
AUTOMOTIVE PRODUCTS (IDENTIFIED AS "XA" IN THE PART NUMBER) ARE NOT
WARRANTED FOR USE IN THE DEPLOYMENT OF AIRBAGS OR FOR USE IN APPLICATIONS
THAT AFFECT CONTROL OF A VEHICLE ("SAFETY APPLICATION") UNLESS THERE IS A
SAFETY CONCEPT OR REDUNDANCY FEATURE CONSISTENT WITH THE ISO 26262
AUTOMOTIVE SAFETY STANDARD ("SAFETY DESIGN"). CUSTOMER SHALL, PRIOR TO USING
OR DISTRIBUTING ANY SYSTEMS THAT INCORPORATE PRODUCTS, THOROUGHLY TEST
SUCH SYSTEMS FOR SAFETY PURPOSES. USE OF PRODUCTS IN A SAFETY APPLICATION
WITHOUT A SAFETY DESIGN IS FULLY AT THE RISK OF CUSTOMER, SUBJECT ONLY TO
APPLICABLE LAWS AND REGULATIONS GOVERNING LIMITATIONS ON PRODUCT
LIABILITY.
Copyright
© Copyright 2015-2022 Advanced Micro Devices, Inc. Xilinx, the Xilinx logo, Alveo, Arx, Kintex,
Kria, Spartan, Versal, Vis, Virtex, Vivado, Zynq, and other designated brands included herein are
trademarks of Xilinx in the United States and other countries. AMBA, AMBA Designer, Arm,
ARM1176JZ-S, CoreSight, Cortex, PrimeCell, Mali, and MPCore are trademarks of Arm Limited in
the EU and other countries. The DisplayPort Icon is a trademark of the Video Electronics
Standards Associaon, registered in the U.S. and other countries. PCI, PCIe, and PCI Express are
trademarks of PCI-SIG and used under license. All other trademarks are the property of their
respecve owners.
Appendix B: Additional Resources and Legal Notices
UG1137 (v2022.2) November 2, 2022 www.xilinx.com
Zynq UltraScale+ MPSoC: Software Developers Guide 268