Skip to content

Conversation

davidkopp
Copy link
Collaborator

@davidkopp davidkopp commented Sep 1, 2025

Introduces a new detection type "runtime". This makes sense to be able to get information e.g. about the java runtime environment and its artifacts (JAR files). Until now we only had detectors that collected "dependencies".

Marked as draft. Have to be discussed with Arne and Didi.

If it should be added, some more refactorings are required, e.g. changing the parent class of JavaRuntimeDetector from PackageManagerDetector to a new base class RuntimeDetector.

Implements comprehensive Java runtime detection for production environments:

* **JavaRuntimeDetector**: Detects JAR files and Java runtime metadata
  - Scans working directory for .jar files recursively
  - Extracts versions from JAR manifests and filenames
  - Collects Java runtime environment information
  - Generates location-based hashes for change detection

* **Runtime Artifact Model**: New schema for deployment artifacts
  - Uses `type: "runtime"` instead of scope-based model
  - `artifacts` field contains deployment files (vs dependencies)
  - `runtime_environment` provides structured platform info
  - Complements existing dependency model for package managers

* **Orchestrator Integration**: Enhanced to handle both models
  - Supports both `dependencies` and `artifacts` detection
  - Updated debug output for artifact counting
  - Maintains backward compatibility

* **Comprehensive Testing**: Docker-based integration tests
  - Tests JAR detection with/without Java runtime
  - Validates version extraction and metadata collection
  - Uses eclipse-temurin:17-jre and alpine:latest images

* **Complete Documentation**:
  - Technical detector documentation with examples
  - Updated project specification with dual models
  - Test documentation and usage guides

Addresses production observability gap where JAR files exist but Maven
build tools are unavailable. Establishes foundation for other runtime
artifact detectors (Go, Python, Node.js, etc.).

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@davidkopp
Copy link
Collaborator Author

davidkopp commented Sep 1, 2025

How does Syft handle Java environments?

TLDR:

  1. Syft collects the manifest information from all dependencies inside the JAR ("java-archive-cataloger") -> should be done by the dependency_resolver as well
  2. Syft also collects the java runtime information with a special cataloger ("java-jvm-cataloger")
  3. Syft collects shared libraries by scanning the filesystem.
  4. Syft uses at the same structure for all artifacts but different metadata attributes categorized by a metadataType. At some level we also need a type information, e.g. to separate the metadata of a package from a runtime (e.g. JVM).

Full example output: https://github.com/green-coding-solutions/dependency-resolver/blob/tool-comparison/docs/tool-comparison/kadai/syft-json_formatted.json

Dependencies are collected by opening the JAR file as a ZIP and collecting the manifest information.
Example output of the collected artifact "commons-lang3":

        {
            "id": "78633107ed603e82",
            "name": "commons-lang3",
            "version": "3.17.0",
            "type": "java-archive",
            "foundBy": "java-archive-cataloger",
            "locations": [
                {
                    "path": "/workspace/BOOT-INF/lib/commons-lang3-3.17.0.jar",
                    "layerID": "sha256:81b57ba95e5cd99f7e1ac41ee5b00c04ca227d971c1c8fd75cec15088b601d08",
                    "accessPath": "/workspace/BOOT-INF/lib/commons-lang3-3.17.0.jar",
                    "annotations": {
                        "evidence": "primary"
                    }
                }
            ],
            "licenses": [
                // ...
            ],
            "language": "java",
            "cpes": [
                {
                    "cpe": "cpe:2.3:a:apache:commons-lang3:3.17.0:*:*:*:*:*:*:*",
                    "source": "syft-generated"
                },
                {
                    "cpe": "cpe:2.3:a:apache:commons_lang3:3.17.0:*:*:*:*:*:*:*",
                    "source": "syft-generated"
                },
                {
                    "cpe": "cpe:2.3:a:apache:commons:3.17.0:*:*:*:*:*:*:*",
                    "source": "syft-generated"
                },
                {
                    "cpe": "cpe:2.3:a:apache:lang3:3.17.0:*:*:*:*:*:*:*",
                    "source": "syft-generated"
                }
            ],
            "purl": "pkg:maven/org.apache.commons/[email protected]",
            "metadataType": "java-archive",
            "metadata": {
                "virtualPath": "/workspace/BOOT-INF/lib/commons-lang3-3.17.0.jar",
                "manifest": {
                    "main": [
                        {
                            "key": "Manifest-Version",
                            "value": "1.0"
                        },
                        {
                            "key": "Created-By",
                            "value": "Apache Maven Bundle Plugin 5.1.9"
                        },
                        {
                            "key": "Build-Jdk-Spec",
                            "value": "17"
                        },
                        {
                            "key": "Specification-Title",
                            "value": "Apache Commons Lang"
                        },
                        {
                            "key": "Specification-Version",
                            "value": "3.17"
                        },
                        {
                            "key": "Specification-Vendor",
                            "value": "The Apache Software Foundation"
                        },
                        {
                            "key": "Implementation-Title",
                            "value": "Apache Commons Lang"
                        },
                        {
                            "key": "Implementation-Version",
                            "value": "3.17.0"
                        },
                        {
                            "key": "Implementation-Vendor",
                            "value": "The Apache Software Foundation"
                        },
                        {
                            "key": "Automatic-Module-Name",
                            "value": "org.apache.commons.lang3"
                        },
                        {
                            "key": "Bundle-Description",
                            "value": "Apache Commons Lang, a package of Java utility classes for the  classes that are in java.lang's hierarchy, or are considered to be so  standard as to justify existence in java.lang.  The code istested using the latest revision of the JDK for supported  LTS releases: 8, 11, 17 and 21 currently.  See https://github.com/apache/commons-lang/blob/master/.github/workflows/maven.yml    Please ensure your buildenvironment is up-to-date and kindly report any build issues."
                        },
                        {
                            "key": "Bundle-DocURL",
                            "value": "https://commons.apache.org/proper/commons-lang/"
                        },
                        {
                            "key": "Bundle-License",
                            "value": "https://www.apache.org/licenses/LICENSE-2.0.txt"
                        },
                        {
                            "key": "Bundle-ManifestVersion",
                            "value": "2"
                        },
                        {
                            "key": "Bundle-Name",
                            "value": "Apache Commons Lang"
                        },
                        {
                            "key": "Bundle-SymbolicName",
                            "value": "org.apache.commons.lang3"
                        },
                        {
                            "key": "Bundle-Vendor",
                            "value": "The Apache Software Foundation"
                        },
                        {
                            "key": "Bundle-Version",
                            "value": "3.17.0"
                        },
                        {
                            "key": "Export-Package",
                            "value": "org.apache.commons.lang3;version=\"3.17.0\",org.apache.commons.lang3.arch;version=\"3.17.0\",org.apache.commons.lang3.builder;version=\"3.17.0\",org.apache.commons.lang3.compare;version=\"3.17.0\",org.apache.commons.lang3.concurrent;version=\"3.17.0\",org.apache.commons.lang3.concurrent.locks;version=\"3.17.0\",org.apache.commons.lang3.event;version=\"3.17.0\",org.apache.commons.lang3.exception;version=\"3.17.0\",org.apache.commons.lang3.function;version=\"3.17.0\",org.apache.commons.lang3.math;version=\"3.17.0\",org.apache.commons.lang3.mutable;version=\"3.17.0\",org.apache.commons.lang3.reflect;version=\"3.17.0\",org.apache.commons.lang3.stream;version=\"3.17.0\",org.apache.commons.lang3.text;version=\"3.17.0\",org.apache.commons.lang3.text.translate;version=\"3.17.0\",org.apache.commons.lang3.time;version=\"3.17.0\",org.apache.commons.lang3.tuple;version=\"3.17.0\",org.apache.commons.lang3.util;version=\"3.17.0\""
                        },
                        {
                            "key": "Include-Resource",
                            "value": "META-INF/LICENSE.txt=LICENSE.txt,META-INF/NOTICE.txt=NOTICE.txt"
                        },
                        {
                            "key": "Require-Capability",
                            "value": "osgi.ee;filter:=\"(&(osgi.ee=JavaSE)(version=1.8))\""
                        },
                        {
                            "key": "Tool",
                            "value": "Bnd-6.4.1.202306080939"
                        },
                        {
                            "key": "Multi-Release",
                            "value": "true"
                        }
                    ]
                },
                "pomProperties": {
                    "path": "META-INF/maven/org.apache.commons/commons-lang3/pom.properties",
                    "name": "",
                    "groupId": "org.apache.commons",
                    "artifactId": "commons-lang3",
                    "version": "3.17.0"
                },
                "digest": [
                    {
                        "algorithm": "sha1",
                        "value": "b17d2136f0460dcc0d2016ceefca8723bdf4ee70"
                    }
                ]
            }
        }

The same dependency is also resolved via the files collector:

{
            "id": "ab041a8b9c9515a2",
            "location": {
                "path": "/workspace/BOOT-INF/lib/commons-lang3-3.17.0.jar",
                "layerID": "sha256:81b57ba95e5cd99f7e1ac41ee5b00c04ca227d971c1c8fd75cec15088b601d08"
            },
            "metadata": {
                "mode": 644,
                "type": "RegularFile",
                "userID": 1001,
                "groupID": 1000,
                "mimeType": "application/jar",
                "size": 673587
            },
            "digests": [
                {
                    "algorithm": "sha1",
                    "value": "b17d2136f0460dcc0d2016ceefca8723bdf4ee70"
                },
                {
                    "algorithm": "sha256",
                    "value": "6ee731df5c8e5a2976a1ca023b6bb320ea8d3539fbe64c8a1d5cb765127c33b4"
                }
            ]
        },

The file collector also collects information of shared libraries.
Example /usr/bin/stat:

{
            "id": "f36caac6b59dedcf",
            "location": {
                "path": "/usr/bin/stat",
                "layerID": "sha256:90a2bf02e851326fc70d05470553ed33e578342d6e06bfa0cfaf331c4079b7e4"
            },
            "metadata": {
                "mode": 755,
                "type": "RegularFile",
                "userID": 0,
                "groupID": 0,
                "mimeType": "application/x-sharedlib",
                "size": 80400
            },
            "digests": [
                {
                    "algorithm": "sha1",
                    "value": "e207843976df98c615975d086660a671761e6a78"
                },
                {
                    "algorithm": "sha256",
                    "value": "9b571b54bd2f17f5fbb841e1886c2d364f5138a02533f4ac3dbfbdaf4dddbea3"
                }
            ],
            "executable": {
                "format": "elf",
                "hasExports": false,
                "hasEntrypoint": true,
                "importedLibraries": [
                    "libselinux.so.1",
                    "libc.so.6"
                ],
                "elfSecurityFeatures": {
                    "symbolTableStripped": true,
                    "stackCanary": true,
                    "nx": true,
                    "relRO": "full",
                    "pie": true,
                    "dso": true,
                    "safeStack": false
                }
            }
        },

The java runtime information were collected with a special "java-jvm-cataloger":

        {
            "id": "382c111ad799cf77",
            "name": "openjdk",
            "version": "21.0.8+12-LTS",
            "type": "binary",
            "foundBy": "java-jvm-cataloger",
            "locations": [
                {
                    "path": "/layers/paketo-buildpacks_bellsoft-liberica/jre/release",
                    "layerID": "sha256:7f4d1070a6e96378ddcfd47af8426b8351e479a61a70a1adfd3c62c4d1854831",
                    "accessPath": "/layers/paketo-buildpacks_bellsoft-liberica/jre/release"
                }
            ],
            "licenses": [],
            "language": "",
            "cpes": [
                {
                    "cpe": "cpe:2.3:a:oracle:openjdk:21.0.8:*:*:*:*:*:*:*",
                    "source": "declared"
                }
            ],
            "purl": "pkg:generic/oracle/[email protected]%2B12-LTS",
            "metadataType": "java-jvm-installation",
            "metadata": {
                "release": {
                    "implementor": "BellSoft",
                    "javaRuntimeVersion": "21.0.8+12-LTS",
                    "javaVersion": "21.0.8",
                    "javaVersionDate": "2025-07-15",
                    "libc": "gnu",
                    "modules": [
                        "java.base",
                        "java.compiler",
                        "java.datatransfer",
                        "java.xml",
                        // ...
                    ],
                    "osArch": "x86_64",
                    "osName": "Linux",
                    "source": ".:git:8997ef8f8129+"
                },
                "files": [
                    "/layers/paketo-buildpacks_bellsoft-liberica/jre/LICENSE",
                    "/layers/paketo-buildpacks_bellsoft-liberica/jre/bin/java",
                    // ...
                ]
            }
        },

@davidkopp davidkopp mentioned this pull request Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant