You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
freifunkist-firmware/scripts/json_add_image_info.py

51 lines
1.3 KiB

#!/usr/bin/env python3
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
from os import getenv
from pathlib import Path
from sys import argv
import hashlib
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
import json
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
if len(argv) != 2:
print("ERROR: JSON info script requires output arg")
exit(1)
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
json_path = Path(argv[1])
bin_dir = Path(getenv("BIN_DIR"))
image_file = bin_dir / getenv("IMAGE_NAME")
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
if not image_file.is_file():
print("Skip JSON creation for non existing image ", image_file)
exit(0)
def get_titles():
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz> (backported from commit 07449f692ce4c4525e946401f4c3ed0cbbc8c4df) Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
4 years ago
return [{"title": getenv("DEVICE_TITLE")}]
device_id = getenv("DEVICE_ID")
image_hash = hashlib.sha256(image_file.read_bytes()).hexdigest()
image_info = {
"metadata_version": 1,
"target": "{}/{}".format(getenv("TARGET"), getenv("SUBTARGET")),
"version_code": getenv("VERSION_CODE"),
"version_number": getenv("VERSION_NUMBER"),
"profiles": {
device_id: {
"image_prefix": getenv("IMAGE_PREFIX"),
"images": [
{
"type": getenv("IMAGE_TYPE"),
"name": getenv("IMAGE_NAME"),
"sha256": image_hash,
}
],
"supported_devices": getenv("SUPPORTED_DEVICES").split(),
"titles": get_titles(),
}
},
}
json_path.write_text(json.dumps(image_info, separators=(",", ":")))