| Age | Commit message (Collapse) | Author |
|---|
|
As we always iterate through the entire die_map when expanding
type strings, recursively processing referenced types in
type_expand_child() is not actually necessary. Furthermore,
the type_string kABI rule added in commit c9083467f7b9
("gendwarfksyms: Add a kABI rule to override type strings") can
fail to override type strings for structures due to a missing
kabi_get_type_string() check in this function.
Fix the issue by dropping the unnecessary recursion and moving
the override check to type_expand(). Note that symbol versions
are otherwise unchanged with this patch.
Fixes: c9083467f7b9 ("gendwarfksyms: Add a kABI rule to override type strings")
Reported-by: Giuliano Procida <gprocida@google.com>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
In rare situations where distributions must make significant
changes to otherwise opaque data structures that have
inadvertently been included in the published ABI, keeping
symbol versions stable using the existing kABI macros can
become tedious.
For example, Android decided to switch to a newer io_uring
implementation in the 5.10 GKI kernel "to resolve a huge number
of potential, and known, problems with the codebase," requiring
"horrible hacks" with genksyms:
"A number of the io_uring structures get used in other core
kernel structures, only as "opaque" pointers, so there is
not any real ABI breakage. But, due to the visibility of
the structures going away, the CRC values of many scheduler
variables and functions were changed."
-- https://r.android.com/2425293
While these specific changes probably could have been hidden
from gendwarfksyms using the existing kABI macros, this may not
always be the case.
Add a last resort kABI rule that allows distribution
maintainers to fully override a type string for a symbol or a
type. Also add a more informative error message in case we find
a non-existent type references when calculating versions.
Suggested-by: Giuliano Procida <gprocida@google.com>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
A data structure can be partially opaque to modules if its
allocation is handled by the core kernel, and modules only need
to access some of its members. In this situation, it's possible
to append new members to the structure without breaking the ABI,
as long as the layout for the original members remains unchanged.
For example, consider the following struct:
struct s {
unsigned long a;
void *p;
};
gendwarfksyms --stable --dump-dies produces the following type
expansion:
variable structure_type s {
member base_type long unsigned int byte_size(8) encoding(7) a
data_member_location(0) ,
member pointer_type {
base_type void
} byte_size(8) p data_member_location(8)
} byte_size(16)
To append new members, we can use the KABI_IGNORE() macro to
hide them from gendwarfksyms --stable:
struct s {
/* old members with unchanged layout */
unsigned long a;
void *p;
/* new members not accessed by modules */
KABI_IGNORE(0, unsigned long n);
};
However, we can't hide the fact that adding new members changes
the struct size, as seen in the updated type string:
variable structure_type s {
member base_type long unsigned int byte_size(8) encoding(7) a
data_member_location(0) ,
member pointer_type {
base_type void
} byte_size(8) p data_member_location(8)
} byte_size(24)
In order to support this use case, add a kABI rule that makes it
possible to override the byte_size attribute for types:
/*
* struct s allocation is handled by the kernel, so
* appending new members without changing the original
* layout won't break the ABI.
*/
KABI_BYTE_SIZE(s, 16);
This results in a type string that's unchanged from the original
and therefore, won't change versions for symbols that reference
the changed structure.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Reduce code duplication by moving kABI rule look-ups to separate
functions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Using dwarf_getscopes_die to resolve fully-qualified names turns out to
be rather slow, and also results in duplicate scopes being processed,
which doesn't help. Simply adding an extra pass to resolve names for all
DIEs before processing exports is noticeably faster.
For the object files with the most exports in a defconfig+Rust build,
the performance improvement is consistently >50%:
rust/bindings.o: 1038 exports
before: 9.5980 +- 0.0183 seconds time elapsed ( +- 0.19% )
after: 4.3116 +- 0.0287 seconds time elapsed ( +- 0.67% )
rust/core.o: 424 exports
before: 5.3584 +- 0.0204 seconds time elapsed ( +- 0.38% )
after: 0.05348 +- 0.00129 seconds time elapsed ( +- 2.42% )
^ Not a mistake.
net/core/dev.o: 190 exports
before: 9.0507 +- 0.0297 seconds time elapsed ( +- 0.33% )
after: 3.2882 +- 0.0165 seconds time elapsed ( +- 0.50% )
rust/kernel.o: 129 exports
before: 6.8571 +- 0.0317 seconds time elapsed ( +- 0.46% )
after: 2.9096 +- 0.0316 seconds time elapsed ( +- 1.09% )
net/core/skbuff.o: 120 exports
before: 5.4805 +- 0.0291 seconds time elapsed ( +- 0.53% )
after: 2.0339 +- 0.0231 seconds time elapsed ( +- 1.14% )
drivers/gpu/drm/display/drm_dp_helper.o: 101 exports
before: 1.7877 +- 0.0187 seconds time elapsed ( +- 1.05% )
after: 0.69245 +- 0.00994 seconds time elapsed ( +- 1.44% )
net/core/sock.o: 97 exports
before: 5.8327 +- 0.0653 seconds time elapsed ( +- 1.12% )
after: 2.0784 +- 0.0291 seconds time elapsed ( +- 1.40% )
drivers/net/phy/phy_device.o: 95 exports
before: 3.0671 +- 0.0371 seconds time elapsed ( +- 1.21% )
after: 1.2127 +- 0.0207 seconds time elapsed ( +- 1.70% )
drivers/pci/pci.o: 93 exports
before: 1.1130 +- 0.0113 seconds time elapsed ( +- 1.01% )
after: 0.4848 +- 0.0127 seconds time elapsed ( +- 2.63% )
kernel/sched/core.o: 83 exports
before: 3.5092 +- 0.0223 seconds time elapsed ( +- 0.64% )
after: 1.1231 +- 0.0145 seconds time elapsed ( +- 1.29% )
Overall, a defconfig+DWARF5 build with gendwarfksyms and Rust is 14.8%
faster with this patch applied on my test system. Without Rust, there's
still a 10.4% improvement in build time when gendwarfksyms is used.
Note that symbol versions are unchanged with this patch.
Suggested-by: Giuliano Procida <gprocida@google.com>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
The compiler may choose not to emit type information in DWARF for
external symbols. Clang, for example, does this for symbols not
defined in the current TU.
To provide a way to work around this issue, add support for
__gendwarfksyms_ptr_<symbol> pointers that force the compiler to emit
the necessary type information in DWARF also for the missing symbols.
Example usage:
#define GENDWARFKSYMS_PTR(sym) \
static typeof(sym) *__gendwarfksyms_ptr_##sym __used \
__section(".discard.gendwarfksyms") = &sym;
extern int external_symbol(void);
GENDWARFKSYMS_PTR(external_symbol);
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel data structures without
affecting symbol versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from
version calculation with #ifndef __GENKSYMS__, which would result
in the symbol version not changing even though the actual type has
changed. When we process precompiled object files, this isn't an
option.
Change union processing to recognize field name prefixes that allow
the user to ignore the union completely during symbol versioning with
a __kabi_ignored prefix in a field name, or to replace the type of a
placeholder field using a __kabi_reserved field name prefix.
For example, assume we want to add a new field to an existing
alignment hole in a data structure, and ignore the new field when
calculating symbol versions:
struct struct1 {
int a;
/* a 4-byte alignment hole */
unsigned long b;
};
To add `int n` to the alignment hole, we can add a union that includes
a __kabi_ignored field that causes gendwarfksyms to ignore the entire
union:
struct struct1 {
int a;
union {
char __kabi_ignored_0;
int n;
};
unsigned long b;
};
With --stable, both structs produce the same symbol version.
Alternatively, when a distribution expects future modification to a
data structure, they can explicitly add reserved fields:
struct struct2 {
long a;
long __kabi_reserved_0; /* reserved for future use */
};
To take the field into use, we can again replace it with a union, with
one of the fields keeping the __kabi_reserved name prefix to indicate
the original type:
struct struct2 {
long a;
union {
long __kabi_reserved_0;
struct {
int b;
int v;
};
};
Here gendwarfksyms --stable replaces the union with the type of the
placeholder field when calculating versions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel without affecting symbol
versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from
version calculation with #ifndef __GENKSYMS__, which would result
in the symbol version not changing even though the actual type has
changed. When we process precompiled object files, this isn't an
option.
To support this use case, add a --stable command line flag that
gates kABI stability features that are not needed in mainline
kernels, but can be useful for distributions, and add support for
kABI rules, which can be used to restrict gendwarfksyms output.
The rules are specified as a set of null-terminated strings stored
in the .discard.gendwarfksyms.kabi_rules section. Each rule consists
of four strings as follows:
"version\0type\0target\0value"
The version string ensures the structure can be changed in a
backwards compatible way. The type string indicates the type of the
rule, and target and value strings contain rule-specific data.
Initially support two simple rules:
1. Declaration-only types
A type declaration can change into a full definition when
additional includes are pulled in to the TU, which changes the
versions of any symbol that references the type. Add support
for defining declaration-only types whose definition is not
expanded during versioning.
2. Ignored enumerators
It's possible to add new enum fields without changing the ABI,
but as the fields are included in symbol versioning, this would
change the versions. Add support for ignoring specific fields.
3. Overridden enumerator values
Add support for overriding enumerator values when calculating
versions. This may be needed when the last field of the enum
is used as a sentinel and new fields must be added before it.
Add examples for using the rules under the examples/ directory.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Calculate symbol versions from the fully expanded type strings in
type_map, and output the versions in a genksyms-compatible format.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Add support for producing genksyms-style symtypes files. Process
die_map to find the longest expansions for each type, and use symtypes
references in type definitions. The basic file format is similar to
genksyms, with two notable exceptions:
1. Type names with spaces (common with Rust) in references are
wrapped in single quotes. E.g.:
s#'core::result::Result<u8, core::num::error::ParseIntError>'
2. The actual type definition is the simple parsed DWARF format we
output with --dump-dies, not the preprocessed C-style format
genksyms produces.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Debugging the DWARF processing can be somewhat challenging, so add
more detailed debugging output for die_map operations. Add the
--dump-die-map flag, which adds color coded tags to the output for
die_map changes.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Expand each structure type only once per exported symbol. This
is necessary to support self-referential structures, which would
otherwise result in infinite recursion, and it's sufficient for
catching ABI changes.
Types defined in .c files are opaque to external users and thus
cannot affect the ABI. Consider type definitions in .c files to
be declarations to prevent opaque types from changing symbol
versions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Recursively expand DWARF structure types, i.e. structs, unions, and
enums. Also include relevant DWARF attributes in type strings to
encode structure layout, for example.
Example output with --dump-dies:
subprogram (
formal_parameter structure_type &str {
member pointer_type {
base_type u8 byte_size(1) encoding(7)
} data_ptr data_member_location(0) ,
member base_type usize byte_size(8) encoding(7) length data_member_location(8)
} byte_size(16) alignment(8) msg
)
-> base_type void
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Add support for expanding DW_TAG_array_type, and the subrange type
indicating array size.
Example source code:
const char *s[34];
Output with --dump-dies:
variable array_type[34] {
pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
} byte_size(8)
}
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Add support for expanding DW_TAG_subroutine_type and the parameters
in DW_TAG_formal_parameter. Use this to also expand subprograms.
Example output with --dump-dies:
subprogram (
formal_parameter pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
}
)
-> base_type unsigned long byte_size(8) encoding(7)
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Add support for expanding DWARF type modifiers, such as pointers,
const values etc., and typedefs. These types all have DW_AT_type
attribute pointing to the underlying type, and thus produce similar
output.
Also add linebreaks and indentation to debugging output to make it
more readable.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Basic types in DWARF repeat frequently and traversing the DIEs using
libdw is relatively slow. Add a simple hashtable based cache for the
processed DIEs.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Start making gendwarfksyms more useful by adding support for
expanding DW_TAG_base_type types and basic DWARF attributes.
Example:
$ echo loops_per_jiffy | \
scripts/gendwarfksyms/gendwarfksyms \
--debug --dump-dies vmlinux.o
...
gendwarfksyms: process_symbol: loops_per_jiffy
variable base_type unsigned long byte_size(8) encoding(7)
...
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
The compiler may choose not to emit type information in DWARF for all
aliases, but it's possible for each alias to be exported separately.
To ensure we find type information for the aliases as well, read
{section, address} tuples from the symbol table and match symbols also
by address.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
|
Add a basic DWARF parser, which uses libdw to traverse the debugging
information in an object file and looks for functions and variables.
In follow-up patches, this will be expanded to produce symbol versions
for CONFIG_MODVERSIONS from DWARF.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
|
