diff options
1229 files changed, 10218 insertions, 4445 deletions
@@ -107,7 +107,8 @@ Asahi Lina <lina+kernel@asahilina.net> <lina@asahilina.net> Ashok Raj Nagarajan <quic_arnagara@quicinc.com> <arnagara@codeaurora.org> Ashwin Chaugule <quic_ashwinc@quicinc.com> <ashwinc@codeaurora.org> Asutosh Das <quic_asutoshd@quicinc.com> <asutoshd@codeaurora.org> -Atish Patra <atishp@atishpatra.org> <atish.patra@wdc.com> +Atish Patra <atish.patra@linux.dev> <atishp@atishpatra.org> +Atish Patra <atish.patra@linux.dev> <atish.patra@wdc.com> Avaneesh Kumar Dwivedi <quic_akdwived@quicinc.com> <akdwived@codeaurora.org> Axel Dyks <xl@xlsigned.net> Axel Lin <axel.lin@gmail.com> @@ -425,6 +426,9 @@ Krzysztof Wilczyński <kwilczynski@kernel.org> <krzysztof.wilczynski@linux.com> Krzysztof Wilczyński <kwilczynski@kernel.org> <kw@linux.com> Kshitiz Godara <quic_kgodara@quicinc.com> <kgodara@codeaurora.org> Kuninori Morimoto <kuninori.morimoto.gx@renesas.com> +Kuniyuki Iwashima <kuniyu@google.com> <kuniyu@amazon.com> +Kuniyuki Iwashima <kuniyu@google.com> <kuniyu@amazon.co.jp> +Kuniyuki Iwashima <kuniyu@google.com> <kuni1840@gmail.com> Kuogee Hsieh <quic_khsieh@quicinc.com> <khsieh@codeaurora.org> Lee Jones <lee@kernel.org> <joneslee@google.com> Lee Jones <lee@kernel.org> <lee.jones@canonical.com> @@ -601,6 +605,12 @@ Paul Mackerras <paulus@ozlabs.org> <paulus@samba.org> Paul Mackerras <paulus@ozlabs.org> <paulus@au1.ibm.com> Paul Moore <paul@paul-moore.com> <paul.moore@hp.com> Paul Moore <paul@paul-moore.com> <pmoore@redhat.com> +Paulo Alcantara <pc@manguebit.org> <pcacjr@zytor.com> +Paulo Alcantara <pc@manguebit.org> <paulo@paulo.ac> +Paulo Alcantara <pc@manguebit.org> <pc@cjr.nz> +Paulo Alcantara <pc@manguebit.org> <palcantara@suse.de> +Paulo Alcantara <pc@manguebit.org> <palcantara@suse.com> +Paulo Alcantara <pc@manguebit.org> <pc@manguebit.com> Pavankumar Kondeti <quic_pkondeti@quicinc.com> <pkondeti@codeaurora.org> Peter A Jonsson <pj@ludd.ltu.se> Peter Oruba <peter.oruba@amd.com> diff --git a/Documentation/arch/riscv/cmodx.rst b/Documentation/arch/riscv/cmodx.rst index 8c48bcff3df9..40ba53bed5df 100644 --- a/Documentation/arch/riscv/cmodx.rst +++ b/Documentation/arch/riscv/cmodx.rst @@ -10,13 +10,45 @@ modified by the program itself. Instruction storage and the instruction cache program must enforce its own synchronization with the unprivileged fence.i instruction. -However, the default Linux ABI prohibits the use of fence.i in userspace -applications. At any point the scheduler may migrate a task onto a new hart. If -migration occurs after the userspace synchronized the icache and instruction -storage with fence.i, the icache on the new hart will no longer be clean. This -is due to the behavior of fence.i only affecting the hart that it is called on. -Thus, the hart that the task has been migrated to may not have synchronized -instruction storage and icache. +CMODX in the Kernel Space +------------------------- + +Dynamic ftrace +--------------------- + +Essentially, dynamic ftrace directs the control flow by inserting a function +call at each patchable function entry, and patches it dynamically at runtime to +enable or disable the redirection. In the case of RISC-V, 2 instructions, +AUIPC + JALR, are required to compose a function call. However, it is impossible +to patch 2 instructions and expect that a concurrent read-side executes them +without a race condition. This series makes atmoic code patching possible in +RISC-V ftrace. Kernel preemption makes things even worse as it allows the old +state to persist across the patching process with stop_machine(). + +In order to get rid of stop_machine() and run dynamic ftrace with full kernel +preemption, we partially initialize each patchable function entry at boot-time, +setting the first instruction to AUIPC, and the second to NOP. Now, atmoic +patching is possible because the kernel only has to update one instruction. +According to Ziccif, as long as an instruction is naturally aligned, the ISA +guarantee an atomic update. + +By fixing down the first instruction, AUIPC, the range of the ftrace trampoline +is limited to +-2K from the predetermined target, ftrace_caller, due to the lack +of immediate encoding space in RISC-V. To address the issue, we introduce +CALL_OPS, where an 8B naturally align metadata is added in front of each +pacthable function. The metadata is resolved at the first trampoline, then the +execution can be derect to another custom trampoline. + +CMODX in the User Space +----------------------- + +Though fence.i is an unprivileged instruction, the default Linux ABI prohibits +the use of fence.i in userspace applications. At any point the scheduler may +migrate a task onto a new hart. If migration occurs after the userspace +synchronized the icache and instruction storage with fence.i, the icache on the +new hart will no longer be clean. This is due to the behavior of fence.i only +affecting the hart that it is called on. Thus, the hart that the task has been +migrated to may not have synchronized instruction storage and icache. There are two ways to solve this problem: use the riscv_flush_icache() syscall, or use the ``PR_RISCV_SET_ICACHE_FLUSH_CTX`` prctl() and emit fence.i in diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index f60bf5991755..2aa9be272d5d 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -271,6 +271,10 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_EXT_ZICBOM`: The Zicbom extension is supported, as ratified in commit 3dd606f ("Create cmobase-v1.0.pdf") of riscv-CMOs. + * :c:macro:`RISCV_HWPROBE_EXT_ZABHA`: The Zabha extension is supported as + ratified in commit 49f49c842ff9 ("Update to Rafified state") of + riscv-zabha. + * :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar values to :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF`, but the key was mistakenly classified as a bitmask rather than a value. @@ -335,3 +339,25 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_KEY_ZICBOM_BLOCK_SIZE`: An unsigned int which represents the size of the Zicbom block in bytes. + +* :c:macro:`RISCV_HWPROBE_KEY_VENDOR_EXT_SIFIVE_0`: A bitmask containing the + sifive vendor extensions that are compatible with the + :c:macro:`RISCV_HWPROBE_BASE_BEHAVIOR_IMA`: base system behavior. + + * SIFIVE + + * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVQMACCDOD`: The Xsfqmaccdod vendor + extension is supported in version 1.1 of SiFive Int8 Matrix Multiplication + Extensions Specification. + + * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVQMACCQOQ`: The Xsfqmaccqoq vendor + extension is supported in version 1.1 of SiFive Int8 Matrix Multiplication + Instruction Extensions Specification. + + * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVFNRCLIPXFQF`: The Xsfvfnrclipxfqf + vendor extension is supported in version 1.0 of SiFive FP32-to-int8 Ranged + Clip Instructions Extensions Specification. + + * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVFWMACCQQQ`: The Xsfvfwmaccqqq + vendor extension is supported in version 1.0 of Matrix Multiply Accumulate + Instruction Extensions Specification.
\ No newline at end of file diff --git a/Documentation/core-api/symbol-namespaces.rst b/Documentation/core-api/symbol-namespaces.rst index 06f766a6aab2..32fc73dc5529 100644 --- a/Documentation/core-api/symbol-namespaces.rst +++ b/Documentation/core-api/symbol-namespaces.rst @@ -6,18 +6,8 @@ The following document describes how to use Symbol Namespaces to structure the export surface of in-kernel symbols exported through the family of EXPORT_SYMBOL() macros. -.. Table of Contents - - === 1 Introduction - === 2 How to define Symbol Namespaces - --- 2.1 Using the EXPORT_SYMBOL macros - --- 2.2 Using the DEFAULT_SYMBOL_NAMESPACE define - === 3 How to use Symbols exported in Namespaces - === 4 Loading Modules that use namespaced Symbols - === 5 Automatically creating MODULE_IMPORT_NS statements - -1. Introduction -=============== +Introduction +============ Symbol Namespaces have been introduced as a means to structure the export surface of the in-kernel API. It allows subsystem maintainers to partition @@ -28,15 +18,18 @@ kernel. As of today, modules that make use of symbols exported into namespaces, are required to import the namespace. Otherwise the kernel will, depending on its configuration, reject loading the module or warn about a missing import. -2. How to define Symbol Namespaces -================================== +Additionally, it is possible to put symbols into a module namespace, strictly +limiting which modules are allowed to use these symbols. + +How to define Symbol Namespaces +=============================== Symbols can be exported into namespace using different methods. All of them are changing the way EXPORT_SYMBOL and friends are instrumented to create ksymtab entries. -2.1 Using the EXPORT_SYMBOL macros -================================== +Using the EXPORT_SYMBOL macros +------------------------------ In addition to the macros EXPORT_SYMBOL() and EXPORT_SYMBOL_GPL(), that allow exporting of kernel symbols to the kernel symbol table, variants of these are @@ -54,8 +47,8 @@ refer to ``NULL``. There is no default namespace if none is defined. ``modpost`` and kernel/module/main.c make use the namespace at build time or module load time, respectively. -2.2 Using the DEFAULT_SYMBOL_NAMESPACE define -============================================= +Using the DEFAULT_SYMBOL_NAMESPACE define +----------------------------------------- Defining namespaces for all symbols of a subsystem can be very verbose and may become hard to maintain. Therefore a default define (DEFAULT_SYMBOL_NAMESPACE) @@ -83,8 +76,24 @@ unit as preprocessor statement. The above example would then read:: within the corresponding compilation unit before the #include for <linux/export.h>. Typically it's placed before the first #include statement. -3. How to use Symbols exported in Namespaces -============================================ +Using the EXPORT_SYMBOL_GPL_FOR_MODULES() macro +----------------------------------------------- + +Symbols exported using this macro are put into a module namespace. This +namespace cannot be imported. + +The macro takes a comma separated list of module names, allowing only those +modules to access this symbol. Simple tail-globs are supported. + +For example:: + + EXPORT_SYMBOL_GPL_FOR_MODULES(preempt_notifier_inc, "kvm,kvm-*") + +will limit usage of this symbol to modules whoes name matches the given +patterns. + +How to use Symbols exported in Namespaces +========================================= In order to use symbols that are exported into namespaces, kernel modules need to explicitly import these namespaces. Otherwise the kernel might reject to @@ -106,11 +115,10 @@ inspected with modinfo:: It is advisable to add the MODULE_IMPORT_NS() statement close to other module -metadata definitions like MODULE_AUTHOR() or MODULE_LICENSE(). Refer to section -5. for a way to create missing import statements automatically. +metadata definitions like MODULE_AUTHOR() or MODULE_LICENSE(). -4. Loading Modules that use namespaced Symbols -============================================== +Loading Modules that use namespaced Symbols +=========================================== At module loading time (e.g. ``insmod``), the kernel will check each symbol referenced from the module for its availability and whether the namespace it @@ -121,8 +129,8 @@ allow loading of modules that don't satisfy this precondition, a configuration option is available: Setting MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=y will enable loading regardless, but will emit a warning. -5. Automatically creating MODULE_IMPORT_NS statements -===================================================== +Automatically creating MODULE_IMPORT_NS statements +================================================== Missing namespaces imports can easily be detected at build time. In fact, modpost will emit a warning if a module uses a symbol from a namespace @@ -154,3 +162,6 @@ in-tree modules:: You can also run nsdeps for external module builds. A typical usage is:: $ make -C <path_to_kernel_src> M=$PWD nsdeps + +Note: it will happily generate an import statement for the module namespace; +which will not work and generates build and runtime failures. diff --git a/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-aon-pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-aon-pinctrl.yaml index b470901f5f56..4dbef86bd958 100644 --- a/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-aon-pinctrl.yaml +++ b/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-aon-pinctrl.yaml @@ -15,7 +15,7 @@ description: | Some peripherals such as PWM have their I/O go through the 4 "GPIOs". maintainers: - - Jianlong Huang <jianlong.huang@starfivetech.com> + - Hal Feng <hal.feng@starfivetech.com> properties: compatible: diff --git a/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-sys-pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-sys-pinctrl.yaml index 222b9e240f8a..e2a25a20f6a6 100644 --- a/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-sys-pinctrl.yaml +++ b/Documentation/devicetree/bindings/pinctrl/starfive,jh7110-sys-pinctrl.yaml @@ -18,7 +18,7 @@ description: | any GPIO can be set up to be controlled by any of the peripherals. maintainers: - - Jianlong Huang <jianlong.huang@starfivetech.com> + - Hal Feng <hal.feng@starfivetech.com> properties: compatible: diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Documentation/devicetree/bindings/riscv/extensions.yaml index bcab59e0cc2e..ede6a58ccf53 100644 --- a/Documentation/devicetree/bindings/riscv/extensions.yaml +++ b/Documentation/devicetree/bindings/riscv/extensions.yaml @@ -662,6 +662,31 @@ properties: Registers in the AX45MP datasheet. https://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf + # SiFive + - const: xsfvqmaccdod + description: + SiFive Int8 Matrix Multiplication Extensions Specification. + See more details in + https://www.sifive.com/document-file/sifive-int8-matrix-multiplication-extensions-specification + + - const: xsfvqmaccqoq + description: + SiFive Int8 Matrix Multiplication Extensions Specification. + See more details in + https://www.sifive.com/document-file/sifive-int8-matrix-multiplication-extensions-specification + + - const: xsfvfnrclipxfqf + description: + SiFive FP32-to-int8 Ranged Clip Instructions Extensions Specification. + See more details in + https://www.sifive.com/document-file/fp32-to-int8-ranged-clip-instructions + + - const: xsfvfwmaccqqq + description: + SiFive Matrix Multiply Accumulate Instruction Extensions Specification. + See more details in + https://www.sifive.com/document-file/matrix-multiply-accumulate-instruction + # T-HEAD - const: xtheadvector description: diff --git a/Documentation/features/core/mseal_sys_mappings/arch-support.txt b/Documentation/features/core/mseal_sys_mappings/arch-support.txt index c6cab9760d57..a3c24233eb9b 100644 --- a/Documentation/features/core/mseal_sys_mappings/arch-support.txt +++ b/Documentation/features/core/mseal_sys_mappings/arch-support.txt @@ -12,7 +12,7 @@ | arm64: | ok | | csky: | N/A | | hexagon: | N/A | - | loongarch: | TODO | + | loongarch: | ok | | m68k: | N/A | | microblaze: | N/A | | mips: | TODO | diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst index 2db379b4b31e..4133a336486d 100644 --- a/Documentation/filesystems/overlayfs.rst +++ b/Documentation/filesystems/overlayfs.rst @@ -443,6 +443,13 @@ Only the data of the files in the "data-only" lower layers may be visible when a "metacopy" file in one of the lower layers above it, has a "redirect" to the absolute path of the "lower data" file in the "data-only" lower layer. +Instead of explicitly enabling "metacopy=on" it is sufficient to specify at +least one data-only layer to enable redirection of data to a data-only layer. +In this case other forms of metacopy are rejected. Note: this way data-only +layers may be used toghether with "userxattr", in which case careful attention +must be given to privileges needed to change the "user.overlay.redirect" xattr +to prevent misuse. + Since kernel version v6.8, "data-only" lower layers can also be added using the "datadir+" mount options and the fsconfig syscall from new mount api. For example:: diff --git a/Documentation/filesystems/smb/index.rst b/Documentation/filesystems/smb/index.rst index 1c8597a679ab..6df23b0e45c8 100644 --- a/Documentation/filesystems/smb/index.rst +++ b/Documentation/filesystems/smb/index.rst @@ -8,3 +8,4 @@ CIFS ksmbd cifsroot + smbdirect diff --git a/Documentation/filesystems/smb/smbdirect.rst b/Documentation/filesystems/smb/smbdirect.rst new file mode 100644 index 000000000000..ca6927c0b2c0 --- /dev/null +++ b/Documentation/filesystems/smb/smbdirect.rst @@ -0,0 +1,103 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=========================== +SMB Direct - SMB3 over RDMA +=========================== + +This document describes how to set up the Linux SMB client and server to +use RDMA. + +Overview +======== +The Linux SMB kernel client supports SMB Direct, which is a transport +scheme for SMB3 that uses RDMA (Remote Direct Memory Access) to provide +high throughput and low latencies by bypassing the traditional TCP/IP +stack. +SMB Direct on the Linux SMB client can be tested against KSMBD - a +kernel-space SMB server. + +Installation +============= +- Install an RDMA device. As long as the RDMA device driver is supported + by the kernel, it should work. This includes both software emulators (soft + RoCE, soft iWARP) and hardware devices (InfiniBand, RoCE, iWARP). + +- Install a kernel with SMB Direct support. The first kernel release to + support SMB Direct on both the client and server side is 5.15. Therefore, + a distribution compatible with kernel 5.15 or later is required. + +- Install cifs-utils, which provides the `mount.cifs` command to mount SMB + shares. + +- Configure the RDMA stack + + Make sure that your kernel configuration has RDMA support enabled. Under + Device Drivers -> Infiniband support, update the kernel configuration to + enable Infiniband support. + + Enable the appropriate IB HCA support or iWARP adapter support, + depending on your hardware. + + If you are using InfiniBand, enable IP-over-InfiniBand support. + + For soft RDMA, enable either the soft iWARP (`RDMA _SIW`) or soft RoCE + (`RDMA_RXE`) module. Install the `iproute2` package and use the + `rdma link add` command to load the module and create an + RDMA interface. + + e.g. if your local ethernet interface is `eth0`, you can use: + + .. code-block:: bash + + sudo rdma link add siw0 type siw netdev eth0 + +- Enable SMB Direct support for both the server and the client in the kernel + configuration. + + Server Setup + + .. code-block:: text + + Network File Systems ---> + <M> SMB3 server support + [*] Support for SMB Direct protocol + + Client Setup + + .. code-block:: text + + Network File Systems ---> + <M> SMB3 and CIFS support (advanced network filesystem) + [*] SMB Direct support + +- Build and install the kernel. SMB Direct support will be enabled in the + cifs.ko and ksmbd.ko modules. + +Setup and Usage +================ + +- Set up and start a KSMBD server as described in the `KSMBD documentation + <https://www.kernel.org/doc/Documentation/filesystems/smb/ksmbd.rst>`_. + Also add the "server multi channel support = yes" parameter to ksmbd.conf. + +- On the client, mount the share with `rdma` mount option to use SMB Direct + (specify a SMB version 3.0 or higher using `vers`). + + For example: + + .. code-block:: bash + + mount -t cifs //server/share /mnt/point -o vers=3.1.1,rdma + +- To verify that the mount is using SMB Direct, you can check dmesg for the + following log line after mounting: + + .. code-block:: text + + CIFS: VFS: RDMA transport established + + Or, verify `rdma` mount option for the share in `/proc/mounts`: + + .. code-block:: bash + + cat /proc/mounts | grep cifs diff --git a/Documentation/kbuild/gendwarfksyms.rst b/Documentation/kbuild/gendwarfksyms.rst index e4beaae7e456..ed366250a54e 100644 --- a/Documentation/kbuild/gendwarfksyms.rst +++ b/Documentation/kbuild/gendwarfksyms.rst @@ -2,8 +2,8 @@ DWARF module versioning ======================= -1. Introduction -=============== +Introduction +============ When CONFIG_MODVERSIONS is enabled, symbol versions for modules are typically calculated from preprocessed source code using the @@ -14,8 +14,8 @@ selected, **gendwarfksyms** is used instead to calculate symbol versions from the DWARF debugging information, which contains the necessary details about the final module ABI. -1.1. Usage -========== +Usage +----- gendwarfksyms accepts a list of object files on the command line, and a list of symbol names (one per line) in standard input:: @@ -33,8 +33,8 @@ list of symbol names (one per line) in standard input:: -h, --help Print this message -2. Type information availability -================================ +Type information availability +============================= While symbols are typically exported in the same translation unit (TU) where they're defined, it's also perfectly fine for a TU to export @@ -56,8 +56,8 @@ type for calculating symbol versions even if the symbol is defined elsewhere. The name of the symbol pointer is expected to start with `__gendwarfksyms_ptr_`, followed by the name of the exported symbol. -3. Symtypes output format -========================= +Symtypes output format +====================== Similarly to genksyms, gendwarfksyms supports writing a symtypes file for each processed object that contain types for exported @@ -85,8 +85,8 @@ produces C-style type strings, gendwarfksyms uses the same simple parsed DWARF format produced by **--dump-dies**, but with type references instead of fully expanded strings. -4. Maintaining a stable kABI -============================ +Maintaining a stable kABI +========================= Distribution maintainers often need the ability to make ABI compatible changes to kernel data structures due to LTS updates or backports. Using @@ -104,8 +104,8 @@ for source code annotation. Note that as these features are only used to transform the inputs for symbol versioning, the user is responsible for ensuring that their changes actually won't break the ABI. -4.1. kABI rules -=============== +kABI rules +---------- kABI rules allow distributions to fine-tune certain parts of gendwarfksyms output and thus control how symbol @@ -125,22 +125,25 @@ the rules. The fields are as follows: qualified name of the DWARF Debugging Information Entry (DIE). - `value`: Provides rule-specific data. -The following helper macro, for example, can be used to specify rules +The following helper macros, for example, can be used to specify rules in the source code:: - #define __KABI_RULE(hint, target, value) \ - static const char __PASTE(__gendwarfksyms_rule_, \ + #define ___KABI_RULE(hint, target, value) \ + static const char __PASTE(__gendwarfksyms_rule_, \ __COUNTER__)[] __used __aligned(1) \ __section(".discard.gendwarfksyms.kabi_rules") = \ - "1\0" #hint "\0" #target "\0" #value + "1\0" #hint "\0" target "\0" value + + #define __KABI_RULE(hint, target, value) \ + ___KABI_RULE(hint, #target, #value) Currently, only the rules discussed in this section are supported, but the format is extensible enough to allow further rules to be added as need arises. -4.1.1. Managing definition visibility -===================================== +Managing definition visibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A declaration can change into a full definition when additional includes are pulled into the translation unit. This changes the versions of any @@ -168,8 +171,8 @@ Example usage:: KABI_DECLONLY(s); -4.1.2. Adding enumerators -========================= +Adding enumerators +~~~~~~~~~~~~~~~~~~ For enums, all enumerators and their values are included in calculating symbol versions, which becomes a problem if we later need to add more @@ -223,8 +226,89 @@ Example usage:: KABI_ENUMERATOR_IGNORE(e, C); KABI_ENUMERATOR_VALUE(e, LAST, 2); -4.3. Adding structure members -============================= +Managing structure size changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A data structure can be partially opaque to modules if its allocation is +handled by the core kernel, and modules only need to access some of its +members. In this situation, it's possible to append new members to the +structure without breaking the ABI, as long as the layout for the original +members remains unchanged. + +To append new members, we can hide them from symbol versioning as +described in section :ref:`Hiding members <hiding_members>`, but we can't +hide the increase in structure size. The `byte_size` rule allows us to +override the structure size used for symbol versioning. + +The rule fields are expected to be as follows: + +- `type`: "byte_size" +- `target`: The fully qualified name of the target data structure + (as shown in **--dump-dies** output). +- `value`: A positive decimal number indicating the structure size + in bytes. + +Using the `__KABI_RULE` macro, this rule can be defined as:: + + #define KABI_BYTE_SIZE(fqn, value) \ + __KABI_RULE(byte_size, fqn, value) + +Example usage:: + + struct s { + /* Unchanged original members */ + unsigned long a; + void *p; + + /* Appended new members */ + KABI_IGNORE(0, unsigned long n); + }; + + KABI_BYTE_SIZE(s, 16); + +Overriding type strings +~~~~~~~~~~~~~~~~~~~~~~~ + +In rare situations where distributions must make significant changes to +otherwise opaque data structures that have inadvertently been included +in the published ABI, keeping symbol versions stable using the more +targeted kABI rules can become tedious. The `type_string` rule allows us +to override the full type string for a type or a symbol, and even add +types for versioning that no longer exist in the kernel. + +The rule fields are expected to be as follows: + +- `type`: "type_string" +- `target`: The fully qualified name of the target data structure + (as shown in **--dump-dies** output) or symbol. +- `value`: A valid type string (as shown in **--symtypes**) output) + to use instead of the real type. + +Using the `__KABI_RULE` macro, this rule can be defined as:: + + #define KABI_TYPE_STRING(type, str) \ + ___KABI_RULE("type_string", type, str) + +Example usage:: + + /* Override type for a structure */ + KABI_TYPE_STRING("s#s", + "structure_type s { " + "member base_type int byte_size(4) " + "encoding(5) n " + "data_member_location(0) " + "} byte_size(8)"); + + /* Override type for a symbol */ + KABI_TYPE_STRING("my_symbol", "variable s#s"); + +The `type_string` rule should be used only as a last resort if maintaining +a stable symbol versions cannot be reasonably achieved using other +means. Overriding a type string increases the risk of actual ABI breakages +going unnoticed as it hides all changes to the type. + +Adding structure members +------------------------ Perhaps the most common ABI compatible change is adding a member to a kernel data structure. When changes to a structure are anticipated, @@ -237,8 +321,8 @@ natural method. This section describes gendwarfksyms support for using reserved space in data structures and hiding members that don't change the ABI when calculating symbol versions. -4.3.1. Reserving space and replacing members -============================================ +Reserving space and replacing members +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Space is typically reserved for later use by appending integer types, or arrays, to the end of the data structure, but any type can be used. Each @@ -276,8 +360,10 @@ The examples include `KABI_(RESERVE|USE|REPLACE)*` macros that help simplify the process and also ensure the replacement member is correctly aligned and its size won't exceed the reserved space. -4.3.2. Hiding members -===================== +.. _hiding_members: + +Hiding members +~~~~~~~~~~~~~~ Predicting which structures will require changes during the support timeframe isn't always possible, in which case one might have to resort @@ -305,4 +391,5 @@ member to a union where one of the fields has a name starting with unsigned long b; }; -With **--stable**, both versions produce the same symbol version. +With **--stable**, both versions produce the same symbol version. The +examples include a `KABI_IGNORE` macro to simplify the code. diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 38cc656fac20..8aef3650c1f3 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -291,6 +291,10 @@ Example:: # arch/x86/kernel/Makefile extra-y += vmlinux.lds +extra-y is now deprecated because this is equivalent to: + + always-$(KBUILD_BUILTIN) += vmlinux.lds + $(extra-y) should only contain targets needed for vmlinux. Kbuild skips extra-y when vmlinux is apparently not a final goal. diff --git a/Documentation/translations/it_IT/core-api/symbol-namespaces.rst b/Documentation/translations/it_IT/core-api/symbol-namespaces.rst index 6ee713988531..baa344f4523a 100644 --- a/Documentation/translations/it_IT/core-api/symbol-namespaces.rst +++ b/Documentation/translations/it_IT/core-api/symbol-namespaces.rst @@ -10,8 +10,8 @@ Questo documento descrive come usare lo spazio dei nomi dei simboli per strutturare quello che viene esportato internamente al kernel grazie alle macro della famiglia EXPORT_SYMBOL(). -1. Introduzione -=============== +Introduzione +============ Lo spazio dei nomi dei simboli è stato introdotto come mezzo per strutturare l'API esposta internamente al kernel. Permette ai manutentori di un @@ -24,15 +24,15 @@ devono prima importare detto spazio. Altrimenti il kernel, a seconda della configurazione, potrebbe rifiutare di caricare il modulo o avvisare l'utente di un'importazione mancante. -2. Come definire uno spazio dei nomi dei simboli -================================================ +Come definire uno spazio dei nomi dei simboli +============================================= I simboli possono essere esportati in spazi dei nomi usando diversi meccanismi. Tutti questi meccanismi cambiano il modo in cui EXPORT_SYMBOL e simili vengono guidati verso la creazione di voci in ksymtab. -2.1 Usare le macro EXPORT_SYMBOL -================================ +Usare le macro EXPORT_SYMBOL +---------------------------- In aggiunta alle macro EXPORT_SYMBOL() e EXPORT_SYMBOL_GPL(), che permettono di esportare simboli del kernel nella rispettiva tabella, ci sono @@ -53,8 +53,8 @@ di base. Il programma ``modpost`` e il codice in kernel/module/main.c usano lo spazio dei nomi, rispettivamente, durante la compilazione e durante il caricamento di un modulo. -2.2 Usare il simbolo di preprocessore DEFAULT_SYMBOL_NAMESPACE -============================================================== +Usare il simbolo di preprocessore DEFAULT_SYMBOL_NAMESPACE +---------------------------------------------------------- Definire lo spazio dei nomi per tutti i simboli di un sottosistema può essere logorante e di difficile manutenzione. Perciò è stato fornito un simbolo @@ -83,8 +83,8 @@ direttamente nei file da compilare. L'esempio precedente diventerebbe:: Questo va messo prima di un qualsiasi uso di EXPORT_SYMBOL. -3. Come usare i simboli esportati attraverso uno spazio dei nomi -================================================================ +Come usare i simboli esportati attraverso uno spazio dei nomi +============================================================= Per usare i simboli esportati da uno spazio dei nomi, i moduli del kernel devono esplicitamente importare il relativo spazio dei nomi; altrimenti @@ -108,12 +108,10 @@ modinfo:: Si consiglia di posizionare la dichiarazione MODULE_IMPORT_NS() vicino -ai metadati del modulo come MODULE_AUTHOR() o MODULE_LICENSE(). Fate -riferimento alla sezione 5. per creare automaticamente le importazioni -mancanti. +ai metadati del modulo come MODULE_AUTHOR() o MODULE_LICENSE(). -4. Caricare moduli che usano simboli provenienti da spazi dei nomi -================================================================== +Caricare moduli che usano simboli provenienti da spazi dei nomi +=============================================================== Quando un modulo viene caricato (per esempio usando ``insmod``), il kernel verificherà la disponibilità di ogni simbolo usato e se lo spazio dei nomi @@ -125,8 +123,8 @@ un'opzione di configurazione: impostare MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=y caricherà i moduli comunque ma emetterà un avviso. -5. Creare automaticamente la dichiarazione MODULE_IMPORT_NS -=========================================================== +Creare automaticamente la dichiarazione MODULE_IMPORT_NS +======================================================== La mancanza di un'importazione può essere individuata facilmente al momento della compilazione. Infatti, modpost emetterà un avviso se il modulo usa diff --git a/Documentation/translations/zh_CN/core-api/symbol-namespaces.rst b/Documentation/translations/zh_CN/core-api/symbol-namespaces.rst index b1bec219912d..d9477ccea98f 100644 --- a/Documentation/translations/zh_CN/core-api/symbol-namespaces.rst +++ b/Documentation/translations/zh_CN/core-api/symbol-namespaces.rst @@ -14,18 +14,8 @@ 本文档æ述了如何使用符å·å‘½å空间æ¥æž„é€ é€šè¿‡EXPORT_SYMBOL()系列å®å¯¼å‡ºçš„å†…æ ¸å†…ç¬¦å·çš„导出é¢ã€‚ -.. 目录 - - === 1 简介 - === 2 如何定义符å·å‘½å空间 - --- 2.1 使用EXPORT_SYMBOLå® - --- 2.2 使用DEFAULT_SYMBOL_NAMESPACE定义 - === 3 如何使用命å空间ä¸å¯¼å‡ºçš„ç¬¦å· - === 4 åŠ è½½ä½¿ç”¨å‘½å空间符å·çš„æ¨¡å— - === 5 自动创建MODULE_IMPORT_NS声明 - -1. 简介 -======= +简介 +==== 符å·å‘½å空间已ç»è¢«å¼•å…¥ï¼Œä½œä¸ºæž„é€ å†…æ ¸å†…API的导出é¢çš„一ç§æ‰‹æ®µã€‚它å…许å系统维护者将 他们导出的符å·åˆ’分进独立的命å空间。这对于文档的编写éžå¸¸æœ‰ç”¨ï¼ˆæƒ³æƒ³SUBSYSTEM_DEBUG @@ -33,14 +23,14 @@ 的模å—必须导入命å空间。å¦åˆ™ï¼Œå†…æ ¸å°†æ ¹æ®å…¶é…置，拒ç»åŠ 载该模å—或è¦å‘Šè¯´ç¼ºå°‘ 导入。 -2. 如何定义符å·å‘½å空间 -======================= +如何定义符å·å‘½å空间 +==================== 符å·å¯ä»¥ç”¨ä¸åŒçš„方法导出到命åç©ºé—´ã€‚æ‰€æœ‰è¿™äº›éƒ½åœ¨æ”¹å˜ EXPORT_SYMBOL å’Œä¸Žä¹‹ç±»ä¼¼çš„é‚£äº›å® è¢«æ£€æµ‹åˆ°çš„æ–¹å¼ï¼Œä»¥åˆ›å»º ksymtab æ¡ç›®ã€‚ -2.1 使用EXPORT_SYMBOLå® -======================= +使用EXPORT_SYMBOLå® +------------------- 除了å…è®¸å°†å†…æ ¸ç¬¦å·å¯¼å‡ºåˆ°å†…æ ¸ç¬¦å·è¡¨çš„å®EXPORT_SYMBOL()å’ŒEXPORT_SYMBOL_GPL()之外， 这些å®çš„å˜ä½“还å¯ä»¥å°†ç¬¦å·å¯¼å‡ºåˆ°æŸä¸ªå‘½å空间：EXPORT_SYMBOL_NS() å’Œ EXPORT_SYMBOL_NS_GPL()。 @@ -54,8 +44,8 @@ 导出时未指明命å空间的符å·å°†æŒ‡å‘ ``NULL`` 。如果没有定义命å空间，则默认没有。 ``modpost`` å’Œkernel/module/main.c分别在构建时或模å—åŠ è½½æ—¶ä½¿ç”¨å称空间。 -2.2 使用DEFAULT_SYMBOL_NAMESPACE定义 -==================================== +使用DEFAULT_SYMBOL_NAMESPACE定义 +-------------------------------- 为一个å系统的所有符å·å®šä¹‰å‘½å空间å¯èƒ½ä¼šéžå¸¸å†—长，并å¯èƒ½å˜å¾—éš¾ä»¥ç»´æŠ¤ã€‚å› æ¤ï¼Œæˆ‘ 们æ供了一个默认定义（DEFAULT_SYMBOL_NAMESPACE），如果设置了这个定义， å®ƒå°†æˆ @@ -80,8 +70,8 @@ 应置于相关编译å•å…ƒä¸ä»»ä½• EXPORT_SYMBOL å®ä¹‹å‰ -3. 如何使用命å空间ä¸å¯¼å‡ºçš„ç¬¦å· -=============================== +如何使用命å空间ä¸å¯¼å‡ºçš„ç¬¦å· +============================ 为了使用被导出到命å空间的符å·ï¼Œå†…æ ¸æ¨¡å—需è¦æ˜Žç¡®åœ°å¯¼å…¥è¿™äº›å‘½å空间。 å¦åˆ™å†…æ ¸å¯èƒ½ä¼šæ‹’ç»åŠ 载该模å—。模å—代ç 需è¦ä½¿ç”¨å®MODULE_IMPORT_NSæ¥ @@ -100,11 +90,10 @@ 建议将 MODULE_IMPORT_NS() è¯å¥æ·»åŠ 到é 近其他模å—元数æ®å®šä¹‰çš„地方， -如 MODULE_AUTHOR() 或 MODULE_LICENSE() 。关于自动创建缺失的导入 -è¯å¥çš„方法，请å‚考第5节。 +如 MODULE_AUTHOR() 或 MODULE_LICENSE() 。 -4. åŠ è½½ä½¿ç”¨å‘½å空间符å·çš„æ¨¡å— -============================= +åŠ è½½ä½¿ç”¨å‘½å空间符å·çš„æ¨¡å— +========================== 在模å—åŠ è½½æ—¶ï¼ˆæ¯”å¦‚ ``insmod`` ï¼‰ï¼Œå†…æ ¸å°†æ£€æŸ¥æ¯ä¸ªä»Žæ¨¡å—ä¸å¼•ç”¨çš„符å·æ˜¯å¦å¯ 用，以åŠå®ƒå¯èƒ½è¢«å¯¼å‡ºåˆ°çš„åå—空间是å¦è¢«æ¨¡å—å¯¼å…¥ã€‚å†…æ ¸çš„é»˜è®¤è¡Œä¸ºæ˜¯æ‹’ç» @@ -113,8 +102,8 @@ EINVALæ–¹å¼å¤±è´¥ã€‚è¦å…è®¸åŠ è½½ä¸æ»¡è¶³è¿™ä¸ªå‰ææ¡ä»¶çš„模å—，å¯ä» 设置 MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=y å°†ä½¿åŠ è½½ä¸å—å½±å“，但会 å‘出è¦å‘Šã€‚ -5. 自动创建MODULE_IMPORT_NS声明 -=============================== +自动创建MODULE_IMPORT_NS声明 +============================ 缺少命å空间的导入å¯ä»¥åœ¨æž„å»ºæ—¶å¾ˆå®¹æ˜“è¢«æ£€æµ‹åˆ°ã€‚äº‹å®žä¸Šï¼Œå¦‚æžœä¸€ä¸ªæ¨¡å— ä½¿ç”¨äº†ä¸€ä¸ªå‘½å空间的符å·è€Œæ²¡æœ‰å¯¼å…¥å®ƒï¼Œmodpost会å‘出è¦å‘Šã€‚ diff --git a/Documentation/userspace-api/mseal.rst b/Documentation/userspace-api/mseal.rst index 7195a7f91107..ea9b11a0bd89 100644 --- a/Documentation/userspace-api/mseal.rst +++ b/Documentation/userspace-api/mseal.rst @@ -144,7 +144,7 @@ Use cases architecture. The following architectures currently support this feature: x86-64, arm64, - and s390. + loongarch and s390. WARNING: This feature breaks programs which rely on relocating or unmapping system mappings. Known broken software at the time diff --git a/MAINTAINERS b/MAINTAINERS index 8e109f4436eb..0183c028fa18 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5987,7 +5987,7 @@ X: drivers/clk/clkdev.c COMMON INTERNET FILE SYSTEM CLIENT (CIFS and SMB3) M: Steve French <sfrench@samba.org> M: Steve French <smfrench@gmail.com> -R: Paulo Alcantara <pc@manguebit.com> (DFS, global name space) +R: Paulo Alcantara <pc@manguebit.org> (DFS, global name space) R: Ronnie Sahlberg <ronniesahlberg@gmail.com> (directory leases, sparse files) R: Shyam Prasad N <sprasad@microsoft.com> (multichannel) R: Tom Talpey <tom@talpey.com> (RDMA, smbdirect) @@ -6255,6 +6255,7 @@ F: include/linux/cpuhotplug.h F: include/linux/smpboot.h F: kernel/cpu.c F: kernel/smpboot.* +F: rust/helper/cpu.c F: rust/kernel/cpu.rs CPU IDLE TIME MANAGEMENT FRAMEWORK @@ -9281,7 +9282,7 @@ F: include/linux/iomap.h FILESYSTEMS [NETFS LIBRARY] M: David Howells <dhowells@redhat.com> -M: Paulo Alcantara <pc@manguebit.com> +M: Paulo Alcantara <pc@manguebit.org> L: netfs@lists.linux.dev L: linux-fsdevel@vger.kernel.org S: Supported @@ -13271,7 +13272,7 @@ F: arch/powerpc/kvm/ KERNEL VIRTUAL MACHINE FOR RISC-V (KVM/riscv) M: Anup Patel <anup@brainfault.org> -R: Atish Patra <atishp@atishpatra.org> +R: Atish Patra <atish.patra@linux.dev> L: kvm@vger.kernel.org L: kvm-riscv@lists.infradead.org L: linux-riscv@lists.infradead.org @@ -15819,6 +15820,7 @@ R: Rakie Kim <rakie.kim@sk.com> R: Byungchul Park <byungchul@sk.com> R: Gregory Price <gourry@gourry.net> R: Ying Huang <ying.huang@linux.alibaba.com> +R: Alistair Popple <apopple@nvidia.com> L: linux-mm@kvack.org S: Maintained W: http://www.linux-mm.org @@ -15890,6 +15892,25 @@ S: Maintained F: include/linux/secretmem.h F: mm/secretmem.c +MEMORY MANAGEMENT - SWAP +M: Andrew Morton <akpm@linux-foundation.org> +R: Kemeng Shi <shikemeng@huaweicloud.com> +R: Kairui Song <kasong@tencent.com> +R: Nhat Pham <nphamcs@gmail.com> +R: Baoquan He <bhe@redhat.com> +R: Barry Song <baohua@kernel.org> +R: Chris Li <chrisl@kernel.org> +L: linux-mm@kvack.org +S: Maintained +F: include/linux/swap.h +F: include/linux/swapfile.h +F: include/linux/swapops.h +F: mm/page_io.c +F: mm/swap.c +F: mm/swap.h +F: mm/swap_state.c +F: mm/swapfile.c + MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE) M: Andrew Morton <akpm@linux-foundation.org> M: David Hildenbrand <david@redhat.com> @@ -16728,6 +16749,7 @@ L: linux-mm@kvack.org S: Maintained F: arch/*/include/asm/tlb.h F: include/asm-generic/tlb.h +F: include/trace/events/tlb.h F: mm/mmu_gather.c MN88472 MEDIA DRIVER @@ -17473,7 +17495,7 @@ F: tools/testing/selftests/net/srv6* NETWORKING [TCP] M: Eric Dumazet <edumazet@google.com> M: Neal Cardwell <ncardwell@google.com> -R: Kuniyuki Iwashima <kuniyu@amazon.com> +R: Kuniyuki Iwashima <kuniyu@google.com> L: netdev@vger.kernel.org S: Maintained F: Documentation/networking/net_cachelines/tcp_sock.rst @@ -17503,7 +17525,7 @@ F: net/tls/* NETWORKING [SOCKETS] M: Eric Dumazet <edumazet@google.com> -M: Kuniyuki Iwashima <kuniyu@amazon.com> +M: Kuniyuki Iwashima <kuniyu@google.com> M: Paolo Abeni <pabeni@redhat.com> M: Willem de Bruijn <willemb@google.com> S: Maintained @@ -17518,7 +17540,7 @@ F: net/core/scm.c F: net/socket.c NETWORKING [UNIX SOCKETS] -M: Kuniyuki Iwashima <kuniyu@amazon.com> +M: Kuniyuki Iwashima <kuniyu@google.com> S: Maintained F: include/net/af_unix.h F: include/net/netns/unix.h @@ -21333,7 +21355,7 @@ F: arch/riscv/boot/dts/sifive/ F: arch/riscv/boot/dts/starfive/ RISC-V PMU DRIVERS -M: Atish Patra <atishp@atishpatra.org> +M: Atish Patra <atish.patra@linux.dev> R: Anup Patel <anup@brainfault.org> L: linux-riscv@lists.infradead.org S: Supported @@ -23641,7 +23663,6 @@ F: include/dt-bindings/clock/starfive?jh71*.h STARFIVE JH71X0 PINCTRL DRIVERS M: Emil Renner Berthing <kernel@esmil.dk> -M: Jianlong Huang <jianlong.huang@starfivetech.com> M: Hal Feng <hal.feng@starfivetech.com> L: linux-gpio@vger.kernel.org S: Maintained @@ -25014,10 +25035,8 @@ F: mm/shmem.c TOMOYO SECURITY MODULE M: Kentaro Takeda <takedakn@nttdata.co.jp> M: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> -L: tomoyo-dev-en@lists.osdn.me (subscribers-only, for developers in English) -L: tomoyo-users-en@lists.osdn.me (subscribers-only, for users in English) -L: tomoyo-dev@lists.osdn.me (subscribers-only, for developers in Japanese) -L: tomoyo-users@lists.osdn.me (subscribers-only, for users in Japanese) +L: tomoyo-users_en@lists.sourceforge.net (subscribers-only, English language) +L: tomoyo-users_ja@lists.sourceforge.net (subscribers-only, Japanese language) S: Maintained W: https://tomoyo.sourceforge.net/ F: security/tomoyo/ @@ -1,8 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 -PATCHLEVEL = 15 +PATCHLEVEL = 16 SUBLEVEL = 0 -EXTRAVERSION = +EXTRAVERSION = -rc1 NAME = Baby Opossum Posse # *DOCUMENTATION* @@ -754,7 +754,7 @@ targets := # Normally, just do built-in. KBUILD_MODULES := -KBUILD_BUILTIN := 1 +KBUILD_BUILTIN := y # If we have only "make modules", don't compile built-in objects. ifeq ($(MAKECMDGOALS),modules) @@ -766,11 +766,11 @@ endif # Just "make" or "make all" shall build modules as well ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) - KBUILD_MODULES := 1 + KBUILD_MODULES := y endif ifeq ($(MAKECMDGOALS),) - KBUILD_MODULES := 1 + KBUILD_MODULES := y endif export KBUILD_MODULES KBUILD_BUILTIN @@ -1189,13 +1189,8 @@ export ARCH_LIB := $(filter %/, $(libs-y)) export ARCH_DRIVERS := $(drivers-y) $(drivers-m) # Externally visible symbols (used by link-vmlinux.sh) -KBUILD_VMLINUX_OBJS := ./built-in.a -ifdef CONFIG_MODULES -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y))) +KBUILD_VMLINUX_OBJS := built-in.a $(patsubst %/, %/lib.a, $(filter %/, $(libs-y))) KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y)) -else -KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y)) -endif export KBUILD_VMLINUX_LIBS export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds @@ -1203,7 +1198,7 @@ export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds ifdef CONFIG_TRIM_UNUSED_KSYMS # For the kernel to actually contain only the needed exported symbols, # we have to build modules as well to determine what those symbols are. -KBUILD_MODULES := 1 +KBUILD_MODULES := y endif # '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14 @@ -1370,7 +1365,7 @@ PHONY += archheaders archscripts hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj PHONY += headers -headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts +headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders ifdef HEADER_ARCH $(Q)$(MAKE) -f $(srctree)/Makefile HEADER_ARCH= SRCARCH=$(HEADER_ARCH) headers else @@ -1543,7 +1538,7 @@ all: modules # the built-in objects during the descend as well, in order to # make sure the checksums are up to date before we record them. ifdef CONFIG_MODVERSIONS - KBUILD_BUILTIN := 1 + KBUILD_BUILTIN := y endif # Build modules @@ -1552,7 +1547,7 @@ endif # *.ko are usually independent of vmlinux, but CONFIG_DEBUG_INFO_BTF_MODULES # is an exception. ifdef CONFIG_DEBUG_INFO_BTF_MODULES -KBUILD_BUILTIN := 1 +KBUILD_BUILTIN := y modules: vmlinux endif @@ -1837,9 +1832,12 @@ rustfmtcheck: rustfmt # Misc # --------------------------------------------------------------------------- +# Run misc checks when ${KBUILD_EXTRA_WARN} contains 1 PHONY += misc-check +ifneq ($(findstring 1,$(KBUILD_EXTRA_WARN)),) misc-check: $(Q)$(srctree)/scripts/misc-check +endif all: misc-check @@ -1865,7 +1863,7 @@ filechk_kernel.release = echo $(KERNELRELEASE) # We are always building only modules. KBUILD_BUILTIN := -KBUILD_MODULES := 1 +KBUILD_MODULES := y build-dir := . @@ -1993,7 +1991,7 @@ endif single-goals := $(addprefix $(build-dir)/, $(single-no-ko)) -KBUILD_MODULES := 1 +KBUILD_MODULES := y endif diff --git a/arch/alpha/include/asm/pgtable.h b/arch/alpha/include/asm/pgtable.h index 2676017f42f1..44e7aedac6e8 100644 --- a/arch/alpha/include/asm/pgtable.h +++ b/arch/alpha/include/asm/pgtable.h @@ -327,7 +327,7 @@ extern inline pte_t mk_swap_pte(unsigned long type, unsigned long offset) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/alpha/kernel/Makefile b/arch/alpha/kernel/Makefile index b6c862dff1f6..187cd8df2faf 100644 --- a/arch/alpha/kernel/Makefile +++ b/arch/alpha/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the linux kernel. # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds asflags-y := $(KBUILD_CFLAGS) ccflags-y := -Wno-sign-compare diff --git a/arch/alpha/kernel/srmcons.c b/arch/alpha/kernel/srmcons.c index a89ce84371f9..d19e51ec711d 100644 --- a/arch/alpha/kernel/srmcons.c +++ b/arch/alpha/kernel/srmcons.c @@ -69,7 +69,8 @@ srmcons_do_receive_chars(struct tty_port *port) static void srmcons_receive_chars(struct timer_list *t) { - struct srmcons_private *srmconsp = from_timer(srmconsp, t, timer); + struct srmcons_private *srmconsp = timer_container_of(srmconsp, t, + timer); struct tty_port *port = &srmconsp->port; unsigned long flags; int incr = 10; diff --git a/arch/arc/include/asm/arcregs.h b/arch/arc/include/asm/arcregs.h index 005d9e4d187a..a31bbf5c8bbc 100644 --- a/arch/arc/include/asm/arcregs.h +++ b/arch/arc/include/asm/arcregs.h @@ -144,7 +144,7 @@ #define ARC_AUX_AGU_MOD2 0x5E2 #define ARC_AUX_AGU_MOD3 0x5E3 -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <soc/arc/arc_aux.h> diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h index 592d7fffc223..e615c42b93ba 100644 --- a/arch/arc/include/asm/atomic.h +++ b/arch/arc/include/asm/atomic.h @@ -6,7 +6,7 @@ #ifndef _ASM_ARC_ATOMIC_H #define _ASM_ARC_ATOMIC_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/types.h> #include <linux/compiler.h> @@ -31,6 +31,6 @@ #include <asm/atomic64-arcv2.h> #endif -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/atomic64-arcv2.h b/arch/arc/include/asm/atomic64-arcv2.h index 9b5791b85471..73080a664369 100644 --- a/arch/arc/include/asm/atomic64-arcv2.h +++ b/arch/arc/include/asm/atomic64-arcv2.h @@ -137,12 +137,9 @@ ATOMIC64_OPS(xor, xor, xor) #undef ATOMIC64_OP_RETURN #undef ATOMIC64_OP -static inline s64 -arch_atomic64_cmpxchg(atomic64_t *ptr, s64 expected, s64 new) +static inline u64 __arch_cmpxchg64_relaxed(volatile void *ptr, u64 old, u64 new) { - s64 prev; - - smp_mb(); + u64 prev; __asm__ __volatile__( "1: llockd %0, [%1] \n" @@ -152,14 +149,12 @@ arch_atomic64_cmpxchg(atomic64_t *ptr, s64 expected, s64 new) " bnz 1b \n" "2: \n" : "=&r"(prev) - : "r"(ptr), "ir"(expected), "r"(new) - : "cc"); /* memory clobber comes from smp_mb() */ - - smp_mb(); + : "r"(ptr), "ir"(old), "r"(new) + : "memory", "cc"); return prev; } -#define arch_atomic64_cmpxchg arch_atomic64_cmpxchg +#define arch_cmpxchg64_relaxed __arch_cmpxchg64_relaxed static inline s64 arch_atomic64_xchg(atomic64_t *ptr, s64 new) { diff --git a/arch/arc/include/asm/bitops.h b/arch/arc/include/asm/bitops.h index f5a936496f06..5340c2871392 100644 --- a/arch/arc/include/asm/bitops.h +++ b/arch/arc/include/asm/bitops.h @@ -10,7 +10,7 @@ #error only <linux/bitops.h> can be included directly #endif -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/types.h> #include <linux/compiler.h> @@ -192,6 +192,6 @@ static inline __attribute__ ((const)) unsigned long __ffs(unsigned long x) #include <asm-generic/bitops/le.h> #include <asm-generic/bitops/ext2-atomic-setbit.h> -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/bug.h b/arch/arc/include/asm/bug.h index 4c453ba96c51..171c16021f70 100644 --- a/arch/arc/include/asm/bug.h +++ b/arch/arc/include/asm/bug.h @@ -6,7 +6,7 @@ #ifndef _ASM_ARC_BUG_H #define _ASM_ARC_BUG_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/ptrace.h> @@ -29,6 +29,6 @@ void die(const char *str, struct pt_regs *regs, unsigned long address); #include <asm-generic/bug.h> -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/cache.h b/arch/arc/include/asm/cache.h index f0f1fc5d62b6..040a97f4dd82 100644 --- a/arch/arc/include/asm/cache.h +++ b/arch/arc/include/asm/cache.h @@ -23,7 +23,7 @@ */ #define ARC_UNCACHED_ADDR_SPACE 0xc0000000 -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/build_bug.h> @@ -65,7 +65,7 @@ extern int ioc_enable; extern unsigned long perip_base, perip_end; -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ /* Instruction cache related Auxiliary registers */ #define ARC_REG_IC_BCR 0x77 /* Build Config reg */ diff --git a/arch/arc/include/asm/current.h b/arch/arc/include/asm/current.h index 06be89f6f2f0..03ffd005f3fa 100644 --- a/arch/arc/include/asm/current.h +++ b/arch/arc/include/asm/current.h @@ -9,7 +9,7 @@ #ifndef _ASM_ARC_CURRENT_H #define _ASM_ARC_CURRENT_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #ifdef CONFIG_ARC_CURR_IN_REG @@ -20,6 +20,6 @@ register struct task_struct *curr_arc asm("gp"); #include <asm-generic/current.h> #endif /* ! CONFIG_ARC_CURR_IN_REG */ -#endif /* ! __ASSEMBLY__ */ +#endif /* ! __ASSEMBLER__ */ #endif /* _ASM_ARC_CURRENT_H */ diff --git a/arch/arc/include/asm/dsp-impl.h b/arch/arc/include/asm/dsp-impl.h index cd5636dfeb6f..fd5fdaad90c1 100644 --- a/arch/arc/include/asm/dsp-impl.h +++ b/arch/arc/include/asm/dsp-impl.h @@ -11,7 +11,7 @@ #define DSP_CTRL_DISABLED_ALL 0 -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ /* clobbers r5 register */ .macro DSP_EARLY_INIT diff --git a/arch/arc/include/asm/dsp.h b/arch/arc/include/asm/dsp.h index f496dbc4640b..eeaaf4e4eabd 100644 --- a/arch/arc/include/asm/dsp.h +++ b/arch/arc/include/asm/dsp.h @@ -7,7 +7,7 @@ #ifndef __ASM_ARC_DSP_H #define __ASM_ARC_DSP_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* * DSP-related saved registers - need to be saved only when you are @@ -24,6 +24,6 @@ struct dsp_callee_regs { #endif }; -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif /* __ASM_ARC_DSP_H */ diff --git a/arch/arc/include/asm/dwarf.h b/arch/arc/include/asm/dwarf.h index a0d5ebe1bc3f..1524c5cf8b59 100644 --- a/arch/arc/include/asm/dwarf.h +++ b/arch/arc/include/asm/dwarf.h @@ -6,7 +6,7 @@ #ifndef _ASM_ARC_DWARF_H #define _ASM_ARC_DWARF_H -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ #ifdef ARC_DW2_UNWIND_AS_CFI @@ -38,6 +38,6 @@ #endif /* !ARC_DW2_UNWIND_AS_CFI */ -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* _ASM_ARC_DWARF_H */ diff --git a/arch/arc/include/asm/entry.h b/arch/arc/include/asm/entry.h index 38c35722cebf..f453af251a1a 100644 --- a/arch/arc/include/asm/entry.h +++ b/arch/arc/include/asm/entry.h @@ -13,7 +13,7 @@ #include <asm/processor.h> /* For VMALLOC_START */ #include <asm/mmu.h> -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ #ifdef CONFIG_ISA_ARCOMPACT #include <asm/entry-compact.h> /* ISA specific bits */ @@ -146,7 +146,7 @@ #endif /* CONFIG_ARC_CURR_IN_REG */ -#else /* !__ASSEMBLY__ */ +#else /* !__ASSEMBLER__ */ extern void do_signal(struct pt_regs *); extern void do_notify_resume(struct pt_regs *); diff --git a/arch/arc/include/asm/irqflags-arcv2.h b/arch/arc/include/asm/irqflags-arcv2.h index fb3c21f1a238..30aea562f8aa 100644 --- a/arch/arc/include/asm/irqflags-arcv2.h +++ b/arch/arc/include/asm/irqflags-arcv2.h @@ -50,7 +50,7 @@ #define ISA_INIT_STATUS_BITS (STATUS_IE_MASK | __AD_ENB | \ (ARCV2_IRQ_DEF_PRIO << 1)) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* * Save IRQ state and disable IRQs @@ -170,6 +170,6 @@ static inline void arc_softirq_clear(int irq) seti .endm -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/irqflags-compact.h b/arch/arc/include/asm/irqflags-compact.h index 936a2f21f315..85c2f6bcde0c 100644 --- a/arch/arc/include/asm/irqflags-compact.h +++ b/arch/arc/include/asm/irqflags-compact.h @@ -40,7 +40,7 @@ #define ISA_INIT_STATUS_BITS STATUS_IE_MASK -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /****************************************************************** * IRQ Control Macros @@ -196,6 +196,6 @@ static inline int arch_irqs_disabled(void) flag \scratch .endm -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/jump_label.h b/arch/arc/include/asm/jump_label.h index a339223d9e05..66ead75784d9 100644 --- a/arch/arc/include/asm/jump_label.h +++ b/arch/arc/include/asm/jump_label.h @@ -2,7 +2,7 @@ #ifndef _ASM_ARC_JUMP_LABEL_H #define _ASM_ARC_JUMP_LABEL_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/stringify.h> #include <linux/types.h> @@ -68,5 +68,5 @@ struct jump_entry { jump_label_t key; }; -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/linkage.h b/arch/arc/include/asm/linkage.h index 8a3fb71e9cfa..ba3cb65b5eaa 100644 --- a/arch/arc/include/asm/linkage.h +++ b/arch/arc/include/asm/linkage.h @@ -12,7 +12,7 @@ #define __ALIGN .align 4 #define __ALIGN_STR __stringify(__ALIGN) -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ .macro ST2 e, o, off #ifdef CONFIG_ARC_HAS_LL64 @@ -61,7 +61,7 @@ CFI_ENDPROC ASM_NL \ .size name, .-name -#else /* !__ASSEMBLY__ */ +#else /* !__ASSEMBLER__ */ #ifdef CONFIG_ARC_HAS_ICCM #define __arcfp_code __section(".text.arcfp") @@ -75,6 +75,6 @@ #define __arcfp_data __section(".data") #endif -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/mmu-arcv2.h b/arch/arc/include/asm/mmu-arcv2.h index 41412642f279..5e5482026ac9 100644 --- a/arch/arc/include/asm/mmu-arcv2.h +++ b/arch/arc/include/asm/mmu-arcv2.h @@ -69,7 +69,7 @@ #define PTE_BITS_NON_RWX_IN_PD1 (PAGE_MASK_PHYS | _PAGE_CACHEABLE) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ struct mm_struct; extern int pae40_exist_but_not_enab(void); @@ -100,6 +100,6 @@ static inline void mmu_setup_pgd(struct mm_struct *mm, void *pgd) sr \reg, [ARC_REG_PID] .endm -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/mmu.h b/arch/arc/include/asm/mmu.h index 4ae2db59d494..e3b35ceab582 100644 --- a/arch/arc/include/asm/mmu.h +++ b/arch/arc/include/asm/mmu.h @@ -6,7 +6,7 @@ #ifndef _ASM_ARC_MMU_H #define _ASM_ARC_MMU_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/threads.h> /* NR_CPUS */ diff --git a/arch/arc/include/asm/page.h b/arch/arc/include/asm/page.h index def0dfb95b43..9720fe6b2c24 100644 --- a/arch/arc/include/asm/page.h +++ b/arch/arc/include/asm/page.h @@ -19,7 +19,7 @@ #endif /* CONFIG_ARC_HAS_PAE40 */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #define clear_page(paddr) memset((paddr), 0, PAGE_SIZE) #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) @@ -136,6 +136,6 @@ static inline unsigned long virt_to_pfn(const void *kaddr) #include <asm-generic/memory_model.h> /* page_to_pfn, pfn_to_page */ #include <asm-generic/getorder.h> -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/pgtable-bits-arcv2.h b/arch/arc/include/asm/pgtable-bits-arcv2.h index 8ebec1b21d24..4630c5acca05 100644 --- a/arch/arc/include/asm/pgtable-bits-arcv2.h +++ b/arch/arc/include/asm/pgtable-bits-arcv2.h @@ -75,7 +75,7 @@ * This is to enable COW mechanism */ /* xwr */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #define pte_write(pte) (pte_val(pte) & _PAGE_WRITE) #define pte_dirty(pte) (pte_val(pte) & _PAGE_DIRTY) @@ -130,7 +130,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma, #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } @@ -142,6 +142,6 @@ PTE_BIT_FUNC(swp_clear_exclusive, &= ~(_PAGE_SWP_EXCLUSIVE)); #include <asm/hugepage.h> #endif -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/pgtable-levels.h b/arch/arc/include/asm/pgtable-levels.h index d1ce4b0f1071..c8f9273372c0 100644 --- a/arch/arc/include/asm/pgtable-levels.h +++ b/arch/arc/include/asm/pgtable-levels.h @@ -85,7 +85,7 @@ #define PTRS_PER_PTE BIT(PMD_SHIFT - PAGE_SHIFT) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #if CONFIG_PGTABLE_LEVELS > 3 #include <asm-generic/pgtable-nop4d.h> @@ -181,6 +181,6 @@ #define pmd_leaf(x) (pmd_val(x) & _PAGE_HW_SZ) #endif -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/pgtable.h b/arch/arc/include/asm/pgtable.h index 4cf45a99fd79..bd580e2b62d7 100644 --- a/arch/arc/include/asm/pgtable.h +++ b/arch/arc/include/asm/pgtable.h @@ -19,7 +19,7 @@ */ #define USER_PTRS_PER_PGD (TASK_SIZE / PGDIR_SIZE) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ extern char empty_zero_page[PAGE_SIZE]; #define ZERO_PAGE(vaddr) (virt_to_page(empty_zero_page)) @@ -29,6 +29,6 @@ extern pgd_t swapper_pg_dir[] __aligned(PAGE_SIZE); /* to cope with aliasing VIPT cache */ #define HAVE_ARCH_UNMAPPED_AREA -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif diff --git a/arch/arc/include/asm/processor.h b/arch/arc/include/asm/processor.h index d606658e2fe7..7f7901ac6643 100644 --- a/arch/arc/include/asm/processor.h +++ b/arch/arc/include/asm/processor.h @@ -11,7 +11,7 @@ #ifndef __ASM_ARC_PROCESSOR_H #define __ASM_ARC_PROCESSOR_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/ptrace.h> #include <asm/dsp.h> @@ -66,7 +66,7 @@ extern void start_thread(struct pt_regs * regs, unsigned long pc, extern unsigned int __get_wchan(struct task_struct *p); -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ /* * Default System Memory Map on ARC diff --git a/arch/arc/include/asm/ptrace.h b/arch/arc/include/asm/ptrace.h index cf79df0b2570..f6c052af8f4d 100644 --- a/arch/arc/include/asm/ptrace.h +++ b/arch/arc/include/asm/ptrace.h @@ -10,7 +10,7 @@ #include <uapi/asm/ptrace.h> #include <linux/compiler.h> -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ typedef union { struct { @@ -172,6 +172,6 @@ static inline unsigned long regs_get_register(struct pt_regs *regs, extern int syscall_trace_enter(struct pt_regs *); extern void syscall_trace_exit(struct pt_regs *); -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif /* __ASM_PTRACE_H */ diff --git a/arch/arc/include/asm/switch_to.h b/arch/arc/include/asm/switch_to.h index 1f85de8288b1..5806106a65f9 100644 --- a/arch/arc/include/asm/switch_to.h +++ b/arch/arc/include/asm/switch_to.h @@ -6,7 +6,7 @@ #ifndef _ASM_ARC_SWITCH_TO_H #define _ASM_ARC_SWITCH_TO_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/sched.h> #include <asm/dsp-impl.h> diff --git a/arch/arc/include/asm/thread_info.h b/arch/arc/include/asm/thread_info.h index 12daaf3a61ea..255d2c774219 100644 --- a/arch/arc/include/asm/thread_info.h +++ b/arch/arc/include/asm/thread_info.h @@ -24,7 +24,7 @@ #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) #define THREAD_SHIFT (PAGE_SHIFT << THREAD_SIZE_ORDER) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/thread_info.h> @@ -62,7 +62,7 @@ static inline __attribute_const__ struct thread_info *current_thread_info(void) return (struct thread_info *)(sp & ~(THREAD_SIZE - 1)); } -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ /* * thread information flags diff --git a/arch/arc/include/uapi/asm/ptrace.h b/arch/arc/include/uapi/asm/ptrace.h index 2a6eff57f6dd..3ae832db278c 100644 --- a/arch/arc/include/uapi/asm/ptrace.h +++ b/arch/arc/include/uapi/asm/ptrace.h @@ -14,7 +14,7 @@ #define PTRACE_GET_THREAD_AREA 25 -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* * Userspace ABI: Register state needed by * -ptrace (gdbserver) @@ -53,6 +53,6 @@ struct user_regs_arcv2 { unsigned long r30, r58, r59; }; -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif /* _UAPI__ASM_ARC_PTRACE_H */ diff --git a/arch/arc/kernel/Makefile b/arch/arc/kernel/Makefile index 95fbf9364c67..fa94fff02419 100644 --- a/arch/arc/kernel/Makefile +++ b/arch/arc/kernel/Makefile @@ -26,4 +26,4 @@ ifdef CONFIG_ISA_ARCOMPACT CFLAGS_fpu.o += -mdpfp endif -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds diff --git a/arch/arc/kernel/unwind.c b/arch/arc/kernel/unwind.c index d8969dab12d4..789cfb9ea14e 100644 --- a/arch/arc/kernel/unwind.c +++ b/arch/arc/kernel/unwind.c @@ -241,15 +241,6 @@ static int cmp_eh_frame_hdr_table_entries(const void *p1, const void *p2) return (e1->start > e2->start) - (e1->start < e2->start); } -static void swap_eh_frame_hdr_table_entries(void *p1, void *p2, int size) -{ - struct eh_frame_hdr_table_entry *e1 = p1; - struct eh_frame_hdr_table_entry *e2 = p2; - - swap(e1->start, e2->start); - swap(e1->fde, e2->fde); -} - static void init_unwind_hdr(struct unwind_table *table, void *(*alloc) (unsigned long)) { @@ -345,7 +336,7 @@ static void init_unwind_hdr(struct unwind_table *table, sort(header->table, n, sizeof(*header->table), - cmp_eh_frame_hdr_table_entries, swap_eh_frame_hdr_table_entries); + cmp_eh_frame_hdr_table_entries, NULL); table->hdrsz = hdrSize; smp_wmb(); diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index 7f1c3b4e3e04..86378eec7757 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -301,7 +301,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(swp) __pte((swp).val) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_isset(pte, L_PTE_SWP_EXCLUSIVE); } diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index b3333d070390..afc9de7ef9a1 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -104,4 +104,4 @@ obj-$(CONFIG_HAVE_ARM_SMCCC) += smccc-call.o obj-$(CONFIG_GENERIC_CPU_VULNERABILITIES) += spectre.o -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 6ce2c5173482..5ccca509dff1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1107,14 +1107,36 @@ static inline u64 *___ctxt_sys_reg(const struct kvm_cpu_context *ctxt, int r) #define ctxt_sys_reg(c,r) (*__ctxt_sys_reg(c,r)) u64 kvm_vcpu_apply_reg_masks(const struct kvm_vcpu *, enum vcpu_sysreg, u64); + +#define __vcpu_assign_sys_reg(v, r, val) \ + do { \ + const struct kvm_cpu_context *ctxt = &(v)->arch.ctxt; \ + u64 __v = (val); \ + if (vcpu_has_nv((v)) && (r) >= __SANITISED_REG_START__) \ + __v = kvm_vcpu_apply_reg_masks((v), (r), __v); \ + \ + ctxt_sys_reg(ctxt, (r)) = __v; \ + } while (0) + +#define __vcpu_rmw_sys_reg(v, r, op, val) \ + do { \ + const struct kvm_cpu_context *ctxt = &(v)->arch.ctxt; \ + u64 __v = ctxt_sys_reg(ctxt, (r)); \ + __v op (val); \ + if (vcpu_has_nv((v)) && (r) >= __SANITISED_REG_START__) \ + __v = kvm_vcpu_apply_reg_masks((v), (r), __v); \ + \ + ctxt_sys_reg(ctxt, (r)) = __v; \ + } while (0) + #define __vcpu_sys_reg(v,r) \ - (*({ \ + ({ \ const struct kvm_cpu_context *ctxt = &(v)->arch.ctxt; \ - u64 *__r = __ctxt_sys_reg(ctxt, (r)); \ + u64 __v = ctxt_sys_reg(ctxt, (r)); \ if (vcpu_has_nv((v)) && (r) >= __SANITISED_REG_START__) \ - *__r = kvm_vcpu_apply_reg_masks((v), (r), *__r);\ - __r; \ - })) + __v = kvm_vcpu_apply_reg_masks((v), (r), __v); \ + __v; \ + }) u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg); void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg); diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 88db8a0c0b37..192d86e1cc76 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -563,7 +563,7 @@ static inline pte_t pte_swp_mkexclusive(pte_t pte) return set_pte_bit(pte, __pgprot(PTE_SWP_EXCLUSIVE)); } -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & PTE_SWP_EXCLUSIVE; } diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 71c29a2a2f19..2920b0a51403 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -78,7 +78,7 @@ $(obj)/vdso32-wrap.o: $(obj)/vdso32/vdso.so obj-y += probes/ obj-y += head.o -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds ifeq ($(CONFIG_DEBUG_EFI),y) AFLAGS_head.o += -DVMLINUX_PATH="\"$(realpath $(objtree)/vmlinux)\"" diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index fdbc8beec930..701ea10a63f1 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -108,16 +108,16 @@ static void timer_set_ctl(struct arch_timer_context *ctxt, u32 ctl) switch(arch_timer_ctx_index(ctxt)) { case TIMER_VTIMER: - __vcpu_sys_reg(vcpu, CNTV_CTL_EL0) = ctl; + __vcpu_assign_sys_reg(vcpu, CNTV_CTL_EL0, ctl); break; case TIMER_PTIMER: - __vcpu_sys_reg(vcpu, CNTP_CTL_EL0) = ctl; + __vcpu_assign_sys_reg(vcpu, CNTP_CTL_EL0, ctl); break; case TIMER_HVTIMER: - __vcpu_sys_reg(vcpu, CNTHV_CTL_EL2) = ctl; + __vcpu_assign_sys_reg(vcpu, CNTHV_CTL_EL2, ctl); break; case TIMER_HPTIMER: - __vcpu_sys_reg(vcpu, CNTHP_CTL_EL2) = ctl; + __vcpu_assign_sys_reg(vcpu, CNTHP_CTL_EL2, ctl); break; default: WARN_ON(1); @@ -130,16 +130,16 @@ static void timer_set_cval(struct arch_timer_context *ctxt, u64 cval) switch(arch_timer_ctx_index(ctxt)) { case TIMER_VTIMER: - __vcpu_sys_reg(vcpu, CNTV_CVAL_EL0) = cval; + __vcpu_assign_sys_reg(vcpu, CNTV_CVAL_EL0, cval); break; case TIMER_PTIMER: - __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = cval; + __vcpu_assign_sys_reg(vcpu, CNTP_CVAL_EL0, cval); break; case TIMER_HVTIMER: - __vcpu_sys_reg(vcpu, CNTHV_CVAL_EL2) = cval; + __vcpu_assign_sys_reg(vcpu, CNTHV_CVAL_EL2, cval); break; case TIMER_HPTIMER: - __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = cval; + __vcpu_assign_sys_reg(vcpu, CNTHP_CVAL_EL2, cval); break; default: WARN_ON(1); @@ -1036,7 +1036,7 @@ void kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu) if (vcpu_has_nv(vcpu)) { struct arch_timer_offset *offs = &vcpu_vtimer(vcpu)->offset; - offs->vcpu_offset = &__vcpu_sys_reg(vcpu, CNTVOFF_EL2); + offs->vcpu_offset = __ctxt_sys_reg(&vcpu->arch.ctxt, CNTVOFF_EL2); offs->vm_offset = &vcpu->kvm->arch.timer_data.poffset; } diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 0e4c805e7e89..1a7dab333f55 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -216,9 +216,9 @@ void kvm_debug_set_guest_ownership(struct kvm_vcpu *vcpu) void kvm_debug_handle_oslar(struct kvm_vcpu *vcpu, u64 val) { if (val & OSLAR_EL1_OSLK) - __vcpu_sys_reg(vcpu, OSLSR_EL1) |= OSLSR_EL1_OSLK; + __vcpu_rmw_sys_reg(vcpu, OSLSR_EL1, |=, OSLSR_EL1_OSLK); else - __vcpu_sys_reg(vcpu, OSLSR_EL1) &= ~OSLSR_EL1_OSLK; + __vcpu_rmw_sys_reg(vcpu, OSLSR_EL1, &=, ~OSLSR_EL1_OSLK); preempt_disable(); kvm_arch_vcpu_put(vcpu); diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 7f6e43d25691..8f6c8f57c6b9 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -103,8 +103,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) fp_state.sve_state = vcpu->arch.sve_state; fp_state.sve_vl = vcpu->arch.sve_max_vl; fp_state.sme_state = NULL; - fp_state.svcr = &__vcpu_sys_reg(vcpu, SVCR); - fp_state.fpmr = &__vcpu_sys_reg(vcpu, FPMR); + fp_state.svcr = __ctxt_sys_reg(&vcpu->arch.ctxt, SVCR); + fp_state.fpmr = __ctxt_sys_reg(&vcpu->arch.ctxt, FPMR); fp_state.fp_type = &vcpu->arch.fp_type; if (vcpu_has_sve(vcpu)) diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c index 424a5107cddb..6a2a899a344e 100644 --- a/arch/arm64/kvm/hyp/exception.c +++ b/arch/arm64/kvm/hyp/exception.c @@ -37,7 +37,7 @@ static inline void __vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg) if (unlikely(vcpu_has_nv(vcpu))) vcpu_write_sys_reg(vcpu, val, reg); else if (!__vcpu_write_sys_reg_to_cpu(val, reg)) - __vcpu_sys_reg(vcpu, reg) = val; + __vcpu_assign_sys_reg(vcpu, reg, val); } static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode, @@ -51,7 +51,7 @@ static void __vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long target_mode, } else if (has_vhe()) { write_sysreg_el1(val, SYS_SPSR); } else { - __vcpu_sys_reg(vcpu, SPSR_EL1) = val; + __vcpu_assign_sys_reg(vcpu, SPSR_EL1, val); } } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index bb9f2eecfb67..76dfda116e56 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -45,7 +45,7 @@ static inline void __fpsimd_save_fpexc32(struct kvm_vcpu *vcpu) if (!vcpu_el1_is_32bit(vcpu)) return; - __vcpu_sys_reg(vcpu, FPEXC32_EL2) = read_sysreg(fpexc32_el2); + __vcpu_assign_sys_reg(vcpu, FPEXC32_EL2, read_sysreg(fpexc32_el2)); } static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) @@ -456,7 +456,7 @@ static inline void fpsimd_lazy_switch_to_host(struct kvm_vcpu *vcpu) */ if (vcpu_has_sve(vcpu)) { zcr_el1 = read_sysreg_el1(SYS_ZCR); - __vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)) = zcr_el1; + __vcpu_assign_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu), zcr_el1); /* * The guest's state is always saved using the guest's max VL. diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index b9cff893bbe0..4d0dbea4c56f 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -307,11 +307,11 @@ static inline void __sysreg32_save_state(struct kvm_vcpu *vcpu) vcpu->arch.ctxt.spsr_irq = read_sysreg(spsr_irq); vcpu->arch.ctxt.spsr_fiq = read_sysreg(spsr_fiq); - __vcpu_sys_reg(vcpu, DACR32_EL2) = read_sysreg(dacr32_el2); - __vcpu_sys_reg(vcpu, IFSR32_EL2) = read_sysreg(ifsr32_el2); + __vcpu_assign_sys_reg(vcpu, DACR32_EL2, read_sysreg(dacr32_el2)); + __vcpu_assign_sys_reg(vcpu, IFSR32_EL2, read_sysreg(ifsr32_el2)); if (has_vhe() || kvm_debug_regs_in_use(vcpu)) - __vcpu_sys_reg(vcpu, DBGVCR32_EL2) = read_sysreg(dbgvcr32_el2); + __vcpu_assign_sys_reg(vcpu, DBGVCR32_EL2, read_sysreg(dbgvcr32_el2)); } static inline void __sysreg32_restore_state(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 8e8848de4d47..e9198e56e784 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -26,7 +26,7 @@ void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) { - __vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR); + __vcpu_assign_sys_reg(vcpu, ZCR_EL1, read_sysreg_el1(SYS_ZCR)); /* * On saving/restoring guest sve state, always use the maximum VL for * the guest. The layout of the data when saving the sve state depends @@ -79,7 +79,7 @@ static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) has_fpmr = kvm_has_fpmr(kern_hyp_va(vcpu->kvm)); if (has_fpmr) - __vcpu_sys_reg(vcpu, FPMR) = read_sysreg_s(SYS_FPMR); + __vcpu_assign_sys_reg(vcpu, FPMR, read_sysreg_s(SYS_FPMR)); if (system_supports_sve()) __hyp_sve_restore_host(); diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index c9b330dc2066..09df2b42bc1b 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -223,9 +223,9 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) */ val = read_sysreg_el0(SYS_CNTP_CVAL); if (map.direct_ptimer == vcpu_ptimer(vcpu)) - __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = val; + __vcpu_assign_sys_reg(vcpu, CNTP_CVAL_EL0, val); if (map.direct_ptimer == vcpu_hptimer(vcpu)) - __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = val; + __vcpu_assign_sys_reg(vcpu, CNTHP_CVAL_EL2, val); offset = read_sysreg_s(SYS_CNTPOFF_EL2); diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 3814b0b2c937..73e4bc7fde9e 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -18,17 +18,17 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu) { /* These registers are common with EL1 */ - __vcpu_sys_reg(vcpu, PAR_EL1) = read_sysreg(par_el1); - __vcpu_sys_reg(vcpu, TPIDR_EL1) = read_sysreg(tpidr_el1); - - __vcpu_sys_reg(vcpu, ESR_EL2) = read_sysreg_el1(SYS_ESR); - __vcpu_sys_reg(vcpu, AFSR0_EL2) = read_sysreg_el1(SYS_AFSR0); - __vcpu_sys_reg(vcpu, AFSR1_EL2) = read_sysreg_el1(SYS_AFSR1); - __vcpu_sys_reg(vcpu, FAR_EL2) = read_sysreg_el1(SYS_FAR); - __vcpu_sys_reg(vcpu, MAIR_EL2) = read_sysreg_el1(SYS_MAIR); - __vcpu_sys_reg(vcpu, VBAR_EL2) = read_sysreg_el1(SYS_VBAR); - __vcpu_sys_reg(vcpu, CONTEXTIDR_EL2) = read_sysreg_el1(SYS_CONTEXTIDR); - __vcpu_sys_reg(vcpu, AMAIR_EL2) = read_sysreg_el1(SYS_AMAIR); + __vcpu_assign_sys_reg(vcpu, PAR_EL1, read_sysreg(par_el1)); + __vcpu_assign_sys_reg(vcpu, TPIDR_EL1, read_sysreg(tpidr_el1)); + + __vcpu_assign_sys_reg(vcpu, ESR_EL2, read_sysreg_el1(SYS_ESR)); + __vcpu_assign_sys_reg(vcpu, AFSR0_EL2, read_sysreg_el1(SYS_AFSR0)); + __vcpu_assign_sys_reg(vcpu, AFSR1_EL2, read_sysreg_el1(SYS_AFSR1)); + __vcpu_assign_sys_reg(vcpu, FAR_EL2, read_sysreg_el1(SYS_FAR)); + __vcpu_assign_sys_reg(vcpu, MAIR_EL2, read_sysreg_el1(SYS_MAIR)); + __vcpu_assign_sys_reg(vcpu, VBAR_EL2, read_sysreg_el1(SYS_VBAR)); + __vcpu_assign_sys_reg(vcpu, CONTEXTIDR_EL2, read_sysreg_el1(SYS_CONTEXTIDR)); + __vcpu_assign_sys_reg(vcpu, AMAIR_EL2, read_sysreg_el1(SYS_AMAIR)); /* * In VHE mode those registers are compatible between EL1 and EL2, @@ -46,21 +46,21 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu) * are always trapped, ensuring that the in-memory * copy is always up-to-date. A small blessing... */ - __vcpu_sys_reg(vcpu, SCTLR_EL2) = read_sysreg_el1(SYS_SCTLR); - __vcpu_sys_reg(vcpu, TTBR0_EL2) = read_sysreg_el1(SYS_TTBR0); - __vcpu_sys_reg(vcpu, TTBR1_EL2) = read_sysreg_el1(SYS_TTBR1); - __vcpu_sys_reg(vcpu, TCR_EL2) = read_sysreg_el1(SYS_TCR); + __vcpu_assign_sys_reg(vcpu, SCTLR_EL2, read_sysreg_el1(SYS_SCTLR)); + __vcpu_assign_sys_reg(vcpu, TTBR0_EL2, read_sysreg_el1(SYS_TTBR0)); + __vcpu_assign_sys_reg(vcpu, TTBR1_EL2, read_sysreg_el1(SYS_TTBR1)); + __vcpu_assign_sys_reg(vcpu, TCR_EL2, read_sysreg_el1(SYS_TCR)); if (ctxt_has_tcrx(&vcpu->arch.ctxt)) { - __vcpu_sys_reg(vcpu, TCR2_EL2) = read_sysreg_el1(SYS_TCR2); + __vcpu_assign_sys_reg(vcpu, TCR2_EL2, read_sysreg_el1(SYS_TCR2)); if (ctxt_has_s1pie(&vcpu->arch.ctxt)) { - __vcpu_sys_reg(vcpu, PIRE0_EL2) = read_sysreg_el1(SYS_PIRE0); - __vcpu_sys_reg(vcpu, PIR_EL2) = read_sysreg_el1(SYS_PIR); + __vcpu_assign_sys_reg(vcpu, PIRE0_EL2, read_sysreg_el1(SYS_PIRE0)); + __vcpu_assign_sys_reg(vcpu, PIR_EL2, read_sysreg_el1(SYS_PIR)); } if (ctxt_has_s1poe(&vcpu->arch.ctxt)) - __vcpu_sys_reg(vcpu, POR_EL2) = read_sysreg_el1(SYS_POR); + __vcpu_assign_sys_reg(vcpu, POR_EL2, read_sysreg_el1(SYS_POR)); } /* @@ -70,13 +70,13 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu) */ val = read_sysreg_el1(SYS_CNTKCTL); val &= CNTKCTL_VALID_BITS; - __vcpu_sys_reg(vcpu, CNTHCTL_EL2) &= ~CNTKCTL_VALID_BITS; - __vcpu_sys_reg(vcpu, CNTHCTL_EL2) |= val; + __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS); + __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val); } - __vcpu_sys_reg(vcpu, SP_EL2) = read_sysreg(sp_el1); - __vcpu_sys_reg(vcpu, ELR_EL2) = read_sysreg_el1(SYS_ELR); - __vcpu_sys_reg(vcpu, SPSR_EL2) = read_sysreg_el1(SYS_SPSR); + __vcpu_assign_sys_reg(vcpu, SP_EL2, read_sysreg(sp_el1)); + __vcpu_assign_sys_reg(vcpu, ELR_EL2, read_sysreg_el1(SYS_ELR)); + __vcpu_assign_sys_reg(vcpu, SPSR_EL2, read_sysreg_el1(SYS_SPSR)); } static void __sysreg_restore_vel2_state(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 4a53e4147fb0..5b191f4dc566 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -1757,7 +1757,7 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) out: for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) - (void)__vcpu_sys_reg(vcpu, sr); + __vcpu_rmw_sys_reg(vcpu, sr, |=, 0); return 0; } diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c index 25c29107f13f..b03dbda7f1ab 100644 --- a/arch/arm64/kvm/pmu-emul.c +++ b/arch/arm64/kvm/pmu-emul.c @@ -178,7 +178,7 @@ static void kvm_pmu_set_pmc_value(struct kvm_pmc *pmc, u64 val, bool force) val |= lower_32_bits(val); } - __vcpu_sys_reg(vcpu, reg) = val; + __vcpu_assign_sys_reg(vcpu, reg, val); /* Recreate the perf event to reflect the updated sample_period */ kvm_pmu_create_perf_event(pmc); @@ -204,7 +204,7 @@ void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu, u64 select_idx, u64 val) void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu, u64 select_idx, u64 val) { kvm_pmu_release_perf_event(kvm_vcpu_idx_to_pmc(vcpu, select_idx)); - __vcpu_sys_reg(vcpu, counter_index_to_reg(select_idx)) = val; + __vcpu_assign_sys_reg(vcpu, counter_index_to_reg(select_idx), val); kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu); } @@ -239,7 +239,7 @@ static void kvm_pmu_stop_counter(struct kvm_pmc *pmc) reg = counter_index_to_reg(pmc->idx); - __vcpu_sys_reg(vcpu, reg) = val; + __vcpu_assign_sys_reg(vcpu, reg, val); kvm_pmu_release_perf_event(pmc); } @@ -503,14 +503,14 @@ static void kvm_pmu_counter_increment(struct kvm_vcpu *vcpu, reg = __vcpu_sys_reg(vcpu, counter_index_to_reg(i)) + 1; if (!kvm_pmc_is_64bit(pmc)) reg = lower_32_bits(reg); - __vcpu_sys_reg(vcpu, counter_index_to_reg(i)) = reg; + __vcpu_assign_sys_reg(vcpu, counter_index_to_reg(i), reg); /* No overflow? move on */ if (kvm_pmc_has_64bit_overflow(pmc) ? reg : lower_32_bits(reg)) continue; /* Mark overflow */ - __vcpu_sys_reg(vcpu, PMOVSSET_EL0) |= BIT(i); + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, BIT(i)); if (kvm_pmu_counter_can_chain(pmc)) kvm_pmu_counter_increment(vcpu, BIT(i + 1), @@ -556,7 +556,7 @@ static void kvm_pmu_perf_overflow(struct perf_event *perf_event, perf_event->attr.sample_period = period; perf_event->hw.sample_period = period; - __vcpu_sys_reg(vcpu, PMOVSSET_EL0) |= BIT(idx); + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, BIT(idx)); if (kvm_pmu_counter_can_chain(pmc)) kvm_pmu_counter_increment(vcpu, BIT(idx + 1), @@ -602,7 +602,7 @@ void kvm_pmu_handle_pmcr(struct kvm_vcpu *vcpu, u64 val) kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu); /* The reset bits don't indicate any state, and shouldn't be saved. */ - __vcpu_sys_reg(vcpu, PMCR_EL0) = val & ~(ARMV8_PMU_PMCR_C | ARMV8_PMU_PMCR_P); + __vcpu_assign_sys_reg(vcpu, PMCR_EL0, (val & ~(ARMV8_PMU_PMCR_C | ARMV8_PMU_PMCR_P))); if (val & ARMV8_PMU_PMCR_C) kvm_pmu_set_counter_value(vcpu, ARMV8_PMU_CYCLE_IDX, 0); @@ -779,7 +779,7 @@ void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, u64 data, u64 reg; reg = counter_index_to_evtreg(pmc->idx); - __vcpu_sys_reg(vcpu, reg) = data & kvm_pmu_evtyper_mask(vcpu->kvm); + __vcpu_assign_sys_reg(vcpu, reg, (data & kvm_pmu_evtyper_mask(vcpu->kvm))); kvm_pmu_create_perf_event(pmc); } @@ -914,9 +914,9 @@ void kvm_vcpu_reload_pmu(struct kvm_vcpu *vcpu) { u64 mask = kvm_pmu_implemented_counter_mask(vcpu); - __vcpu_sys_reg(vcpu, PMOVSSET_EL0) &= mask; - __vcpu_sys_reg(vcpu, PMINTENSET_EL1) &= mask; - __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) &= mask; + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, mask); + __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, mask); + __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, mask); kvm_pmu_reprogram_counter_mask(vcpu, mask); } @@ -1038,7 +1038,7 @@ static void kvm_arm_set_nr_counters(struct kvm *kvm, unsigned int nr) u64 val = __vcpu_sys_reg(vcpu, MDCR_EL2); val &= ~MDCR_EL2_HPMN; val |= FIELD_PREP(MDCR_EL2_HPMN, kvm->arch.nr_pmu_counters); - __vcpu_sys_reg(vcpu, MDCR_EL2) = val; + __vcpu_assign_sys_reg(vcpu, MDCR_EL2, val); } } } diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index a6cf2888d150..76c2f0da821f 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -228,7 +228,7 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg) * to reverse-translate virtual EL2 system registers for a * non-VHE guest hypervisor. */ - __vcpu_sys_reg(vcpu, reg) = val; + __vcpu_assign_sys_reg(vcpu, reg, val); switch (reg) { case CNTHCTL_EL2: @@ -263,7 +263,7 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg) return; memory_write: - __vcpu_sys_reg(vcpu, reg) = val; + __vcpu_assign_sys_reg(vcpu, reg, val); } /* CSSELR values; used to index KVM_REG_ARM_DEMUX_ID_CCSIDR */ @@ -605,7 +605,7 @@ static int set_oslsr_el1(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, if ((val ^ rd->val) & ~OSLSR_EL1_OSLK) return -EINVAL; - __vcpu_sys_reg(vcpu, rd->reg) = val; + __vcpu_assign_sys_reg(vcpu, rd->reg, val); return 0; } @@ -791,7 +791,7 @@ static u64 reset_pmu_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) mask |= GENMASK(n - 1, 0); reset_unknown(vcpu, r); - __vcpu_sys_reg(vcpu, r->reg) &= mask; + __vcpu_rmw_sys_reg(vcpu, r->reg, &=, mask); return __vcpu_sys_reg(vcpu, r->reg); } @@ -799,7 +799,7 @@ static u64 reset_pmu_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) static u64 reset_pmevcntr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { reset_unknown(vcpu, r); - __vcpu_sys_reg(vcpu, r->reg) &= GENMASK(31, 0); + __vcpu_rmw_sys_reg(vcpu, r->reg, &=, GENMASK(31, 0)); return __vcpu_sys_reg(vcpu, r->reg); } @@ -811,7 +811,7 @@ static u64 reset_pmevtyper(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) return 0; reset_unknown(vcpu, r); - __vcpu_sys_reg(vcpu, r->reg) &= kvm_pmu_evtyper_mask(vcpu->kvm); + __vcpu_rmw_sys_reg(vcpu, r->reg, &=, kvm_pmu_evtyper_mask(vcpu->kvm)); return __vcpu_sys_reg(vcpu, r->reg); } @@ -819,7 +819,7 @@ static u64 reset_pmevtyper(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) static u64 reset_pmselr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { reset_unknown(vcpu, r); - __vcpu_sys_reg(vcpu, r->reg) &= PMSELR_EL0_SEL_MASK; + __vcpu_rmw_sys_reg(vcpu, r->reg, &=, PMSELR_EL0_SEL_MASK); return __vcpu_sys_reg(vcpu, r->reg); } @@ -835,7 +835,7 @@ static u64 reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) * The value of PMCR.N field is included when the * vCPU register is read via kvm_vcpu_read_pmcr(). */ - __vcpu_sys_reg(vcpu, r->reg) = pmcr; + __vcpu_assign_sys_reg(vcpu, r->reg, pmcr); return __vcpu_sys_reg(vcpu, r->reg); } @@ -907,7 +907,7 @@ static bool access_pmselr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, return false; if (p->is_write) - __vcpu_sys_reg(vcpu, PMSELR_EL0) = p->regval; + __vcpu_assign_sys_reg(vcpu, PMSELR_EL0, p->regval); else /* return PMSELR.SEL field */ p->regval = __vcpu_sys_reg(vcpu, PMSELR_EL0) @@ -1076,7 +1076,7 @@ static int set_pmreg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, u64 va { u64 mask = kvm_pmu_accessible_counter_mask(vcpu); - __vcpu_sys_reg(vcpu, r->reg) = val & mask; + __vcpu_assign_sys_reg(vcpu, r->reg, val & mask); kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu); return 0; @@ -1103,10 +1103,10 @@ static bool access_pmcnten(struct kvm_vcpu *vcpu, struct sys_reg_params *p, val = p->regval & mask; if (r->Op2 & 0x1) /* accessing PMCNTENSET_EL0 */ - __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) |= val; + __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, |=, val); else /* accessing PMCNTENCLR_EL0 */ - __vcpu_sys_reg(vcpu, PMCNTENSET_EL0) &= ~val; + __vcpu_rmw_sys_reg(vcpu, PMCNTENSET_EL0, &=, ~val); kvm_pmu_reprogram_counter_mask(vcpu, val); } else { @@ -1129,10 +1129,10 @@ static bool access_pminten(struct kvm_vcpu *vcpu, struct sys_reg_params *p, if (r->Op2 & 0x1) /* accessing PMINTENSET_EL1 */ - __vcpu_sys_reg(vcpu, PMINTENSET_EL1) |= val; + __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, |=, val); else /* accessing PMINTENCLR_EL1 */ - __vcpu_sys_reg(vcpu, PMINTENSET_EL1) &= ~val; + __vcpu_rmw_sys_reg(vcpu, PMINTENSET_EL1, &=, ~val); } else { p->regval = __vcpu_sys_reg(vcpu, PMINTENSET_EL1); } @@ -1151,10 +1151,10 @@ static bool access_pmovs(struct kvm_vcpu *vcpu, struct sys_reg_params *p, if (p->is_write) { if (r->CRm & 0x2) /* accessing PMOVSSET_EL0 */ - __vcpu_sys_reg(vcpu, PMOVSSET_EL0) |= (p->regval & mask); + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, (p->regval & mask)); else /* accessing PMOVSCLR_EL0 */ - __vcpu_sys_reg(vcpu, PMOVSSET_EL0) &= ~(p->regval & mask); + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=, ~(p->regval & mask)); } else { p->regval = __vcpu_sys_reg(vcpu, PMOVSSET_EL0); } @@ -1185,8 +1185,8 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, if (!vcpu_mode_priv(vcpu)) return undef_access(vcpu, p, r); - __vcpu_sys_reg(vcpu, PMUSERENR_EL0) = - p->regval & ARMV8_PMU_USERENR_MASK; + __vcpu_assign_sys_reg(vcpu, PMUSERENR_EL0, + (p->regval & ARMV8_PMU_USERENR_MASK)); } else { p->regval = __vcpu_sys_reg(vcpu, PMUSERENR_EL0) & ARMV8_PMU_USERENR_MASK; @@ -1237,7 +1237,7 @@ static int set_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, if (!kvm_supports_32bit_el0()) val |= ARMV8_PMU_PMCR_LC; - __vcpu_sys_reg(vcpu, r->reg) = val; + __vcpu_assign_sys_reg(vcpu, r->reg, val); kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu); return 0; @@ -2213,7 +2213,7 @@ static u64 reset_clidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) if (kvm_has_mte(vcpu->kvm)) clidr |= 2ULL << CLIDR_TTYPE_SHIFT(loc); - __vcpu_sys_reg(vcpu, r->reg) = clidr; + __vcpu_assign_sys_reg(vcpu, r->reg, clidr); return __vcpu_sys_reg(vcpu, r->reg); } @@ -2227,7 +2227,7 @@ static int set_clidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, if ((val & CLIDR_EL1_RES0) || (!(ctr_el0 & CTR_EL0_IDC) && idc)) return -EINVAL; - __vcpu_sys_reg(vcpu, rd->reg) = val; + __vcpu_assign_sys_reg(vcpu, rd->reg, val); return 0; } @@ -2404,7 +2404,7 @@ static bool access_sp_el1(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { if (p->is_write) - __vcpu_sys_reg(vcpu, SP_EL1) = p->regval; + __vcpu_assign_sys_reg(vcpu, SP_EL1, p->regval); else p->regval = __vcpu_sys_reg(vcpu, SP_EL1); @@ -2428,7 +2428,7 @@ static bool access_spsr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { if (p->is_write) - __vcpu_sys_reg(vcpu, SPSR_EL1) = p->regval; + __vcpu_assign_sys_reg(vcpu, SPSR_EL1, p->regval); else p->regval = __vcpu_sys_reg(vcpu, SPSR_EL1); @@ -2440,7 +2440,7 @@ static bool access_cntkctl_el12(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { if (p->is_write) - __vcpu_sys_reg(vcpu, CNTKCTL_EL1) = p->regval; + __vcpu_assign_sys_reg(vcpu, CNTKCTL_EL1, p->regval); else p->regval = __vcpu_sys_reg(vcpu, CNTKCTL_EL1); @@ -2454,7 +2454,9 @@ static u64 reset_hcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) if (!cpus_have_final_cap(ARM64_HAS_HCR_NV1)) val |= HCR_E2H; - return __vcpu_sys_reg(vcpu, r->reg) = val; + __vcpu_assign_sys_reg(vcpu, r->reg, val); + + return __vcpu_sys_reg(vcpu, r->reg); } static unsigned int __el2_visibility(const struct kvm_vcpu *vcpu, @@ -2625,7 +2627,7 @@ static bool access_mdcr(struct kvm_vcpu *vcpu, u64_replace_bits(val, hpmn, MDCR_EL2_HPMN); } - __vcpu_sys_reg(vcpu, MDCR_EL2) = val; + __vcpu_assign_sys_reg(vcpu, MDCR_EL2, val); /* * Request a reload of the PMU to enable/disable the counters @@ -2754,7 +2756,7 @@ static int set_imp_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, static u64 reset_mdcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { - __vcpu_sys_reg(vcpu, r->reg) = vcpu->kvm->arch.nr_pmu_counters; + __vcpu_assign_sys_reg(vcpu, r->reg, vcpu->kvm->arch.nr_pmu_counters); return vcpu->kvm->arch.nr_pmu_counters; } @@ -4790,7 +4792,7 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu) r->reset(vcpu, r); if (r->reg >= __SANITISED_REG_START__ && r->reg < NR_SYS_REGS) - (void)__vcpu_sys_reg(vcpu, r->reg); + __vcpu_rmw_sys_reg(vcpu, r->reg, |=, 0); } set_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &kvm->arch.flags); @@ -5012,7 +5014,7 @@ int kvm_sys_reg_set_user(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg, if (r->set_user) { ret = (r->set_user)(vcpu, r, val); } else { - __vcpu_sys_reg(vcpu, r->reg) = val; + __vcpu_assign_sys_reg(vcpu, r->reg, val); ret = 0; } diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h index cc6338d38766..ef97d9fc67cc 100644 --- a/arch/arm64/kvm/sys_regs.h +++ b/arch/arm64/kvm/sys_regs.h @@ -137,7 +137,7 @@ static inline u64 reset_unknown(struct kvm_vcpu *vcpu, { BUG_ON(!r->reg); BUG_ON(r->reg >= NR_SYS_REGS); - __vcpu_sys_reg(vcpu, r->reg) = 0x1de7ec7edbadc0deULL; + __vcpu_assign_sys_reg(vcpu, r->reg, 0x1de7ec7edbadc0deULL); return __vcpu_sys_reg(vcpu, r->reg); } @@ -145,7 +145,7 @@ static inline u64 reset_val(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { BUG_ON(!r->reg); BUG_ON(r->reg >= NR_SYS_REGS); - __vcpu_sys_reg(vcpu, r->reg) = r->val; + __vcpu_assign_sys_reg(vcpu, r->reg, r->val); return __vcpu_sys_reg(vcpu, r->reg); } diff --git a/arch/arm64/kvm/vgic/vgic-v3-nested.c b/arch/arm64/kvm/vgic/vgic-v3-nested.c index 4f6954c30674..d22a8ad7bcc5 100644 --- a/arch/arm64/kvm/vgic/vgic-v3-nested.c +++ b/arch/arm64/kvm/vgic/vgic-v3-nested.c @@ -356,12 +356,12 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu) val = __vcpu_sys_reg(vcpu, ICH_HCR_EL2); val &= ~ICH_HCR_EL2_EOIcount_MASK; val |= (s_cpu_if->vgic_hcr & ICH_HCR_EL2_EOIcount_MASK); - __vcpu_sys_reg(vcpu, ICH_HCR_EL2) = val; - __vcpu_sys_reg(vcpu, ICH_VMCR_EL2) = s_cpu_if->vgic_vmcr; + __vcpu_assign_sys_reg(vcpu, ICH_HCR_EL2, val); + __vcpu_assign_sys_reg(vcpu, ICH_VMCR_EL2, s_cpu_if->vgic_vmcr); for (i = 0; i < 4; i++) { - __vcpu_sys_reg(vcpu, ICH_AP0RN(i)) = s_cpu_if->vgic_ap0r[i]; - __vcpu_sys_reg(vcpu, ICH_AP1RN(i)) = s_cpu_if->vgic_ap1r[i]; + __vcpu_assign_sys_reg(vcpu, ICH_AP0RN(i), s_cpu_if->vgic_ap0r[i]); + __vcpu_assign_sys_reg(vcpu, ICH_AP1RN(i), s_cpu_if->vgic_ap1r[i]); } for_each_set_bit(i, &shadow_if->lr_map, kvm_vgic_global_state.nr_lr) { @@ -370,7 +370,7 @@ void vgic_v3_put_nested(struct kvm_vcpu *vcpu) val &= ~ICH_LR_STATE; val |= s_cpu_if->vgic_lr[i] & ICH_LR_STATE; - __vcpu_sys_reg(vcpu, ICH_LRN(i)) = val; + __vcpu_assign_sys_reg(vcpu, ICH_LRN(i), val); s_cpu_if->vgic_lr[i] = 0; } diff --git a/arch/csky/include/asm/pgtable.h b/arch/csky/include/asm/pgtable.h index b8378431aeff..5a394be09c35 100644 --- a/arch/csky/include/asm/pgtable.h +++ b/arch/csky/include/asm/pgtable.h @@ -200,7 +200,7 @@ static inline pte_t pte_mkyoung(pte_t pte) return pte; } -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/csky/kernel/Makefile b/arch/csky/kernel/Makefile index de1c3472e8f0..a406a4ac2378 100644 --- a/arch/csky/kernel/Makefile +++ b/arch/csky/kernel/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y += head.o entry.o atomic.o signal.o traps.o irq.o time.o vdso.o vdso/ obj-y += power.o syscall.o syscall_table.o setup.o diff --git a/arch/hexagon/include/asm/pgtable.h b/arch/hexagon/include/asm/pgtable.h index 9fbdfdbc539f..fbf24d1d1ca6 100644 --- a/arch/hexagon/include/asm/pgtable.h +++ b/arch/hexagon/include/asm/pgtable.h @@ -387,7 +387,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) (((type & 0x1f) << 1) | \ ((offset & 0x3ffff8) << 10) | ((offset & 0x7) << 7)) }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/hexagon/kernel/Makefile b/arch/hexagon/kernel/Makefile index 3fdf937eb572..8e0fb4a62315 100644 --- a/arch/hexagon/kernel/Makefile +++ b/arch/hexagon/kernel/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y += head.o obj-$(CONFIG_SMP) += smp.o diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 1a2cf012b8f2..4b19f93379a1 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -69,6 +69,7 @@ config LOONGARCH select ARCH_SUPPORTS_INT128 if CC_HAS_INT128 select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN + select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS select ARCH_SUPPORTS_NUMA_BALANCING select ARCH_SUPPORTS_RT select ARCH_USE_BUILTIN_BSWAP @@ -123,6 +124,7 @@ config LOONGARCH select HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET select HAVE_ARCH_SECCOMP select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_STACKLEAK select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE select HAVE_ARCH_USERFAULTFD_MINOR if USERFAULTFD @@ -187,6 +189,7 @@ config LOONGARCH select MODULES_USE_ELF_RELA if MODULES select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK + select NUMA_MEMBLKS if NUMA select OF select OF_EARLY_FLATTREE select PCI @@ -456,6 +459,15 @@ config SCHED_SMT Improves scheduler's performance when there are multiple threads in one physical core. +config SCHED_MC + bool "Multi-core scheduler support" + depends on SMP + default y + help + Multi-core scheduler support improves the CPU scheduler's decision + making when dealing with multi-core CPU chips at a cost of slightly + increased overhead in some places. + config SMP bool "Multi-Processing support" help @@ -485,10 +497,10 @@ config HOTPLUG_CPU Say N if you want to disable CPU hotplug. config NR_CPUS - int "Maximum number of CPUs (2-256)" - range 2 256 + int "Maximum number of CPUs (2-2048)" + range 2 2048 + default "2048" depends on SMP - default "64" help This allows you to specify the maximum number of CPUs which this kernel will support. diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile index 0304eabbe606..b0703a4e02a2 100644 --- a/arch/loongarch/Makefile +++ b/arch/loongarch/Makefile @@ -181,11 +181,14 @@ vmlinux.elf vmlinux.efi vmlinuz.efi: vmlinux $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $(boot)/$@ install: - $(Q)install -D -m 755 $(KBUILD_IMAGE) $(INSTALL_PATH)/$(image-name-y)-$(KERNELRELEASE) - $(Q)install -D -m 644 .config $(INSTALL_PATH)/config-$(KERNELRELEASE) - $(Q)install -D -m 644 System.map $(INSTALL_PATH)/System.map-$(KERNELRELEASE) + $(call cmd,install) define archhelp - echo ' install - install kernel into $(INSTALL_PATH)' + echo ' vmlinux.elf - Uncompressed ELF kernel image (arch/loongarch/boot/vmlinux.elf)' + echo ' vmlinux.efi - Uncompressed EFI kernel image (arch/loongarch/boot/vmlinux.efi)' + echo ' vmlinuz.efi - GZIP/ZSTD-compressed EFI kernel image (arch/loongarch/boot/vmlinuz.efi)' + echo ' Default when CONFIG_EFI_ZBOOT=y' + echo ' install - Install kernel using (your) ~/bin/$(INSTALLKERNEL) or' + echo ' (distribution) /sbin/$(INSTALLKERNEL) or install.sh to $$(INSTALL_PATH)' echo endef diff --git a/arch/loongarch/boot/dts/loongson-2k0500.dtsi b/arch/loongarch/boot/dts/loongson-2k0500.dtsi index 3b38ff8853a7..760c60eebb89 100644 --- a/arch/loongarch/boot/dts/loongson-2k0500.dtsi +++ b/arch/loongarch/boot/dts/loongson-2k0500.dtsi @@ -169,6 +169,166 @@ interrupts = <3>; }; + pwm@1ff5c000 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c000 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c010 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c010 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c020 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c020 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c030 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c030 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c040 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c040 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c050 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c050 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c060 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c060 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c070 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c070 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c080 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c080 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c090 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c090 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0a0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0a0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0b0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0b0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0c0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0c0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0d0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0d0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0e0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0e0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1ff5c0f0 { + compatible = "loongson,ls2k0500-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1ff5c0f0 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + gmac0: ethernet@1f020000 { compatible = "snps,dwmac-3.70a"; reg = <0x0 0x1f020000 0x0 0x10000>; diff --git a/arch/loongarch/boot/dts/loongson-2k1000-ref.dts b/arch/loongarch/boot/dts/loongson-2k1000-ref.dts index 3514ea78f525..78ea995abf1c 100644 --- a/arch/loongarch/boot/dts/loongson-2k1000-ref.dts +++ b/arch/loongarch/boot/dts/loongson-2k1000-ref.dts @@ -5,6 +5,7 @@ /dts-v1/; +#include "dt-bindings/thermal/thermal.h" #include "loongson-2k1000.dtsi" / { @@ -38,6 +39,13 @@ linux,cma-default; }; }; + + fan0: pwm-fan { + compatible = "pwm-fan"; + cooling-levels = <255 153 85 25>; + pwms = <&pwm1 0 100000 0>; + #cooling-cells = <2>; + }; }; &gmac0 { @@ -92,6 +100,22 @@ #size-cells = <0>; }; +&pwm1 { + status = "okay"; + + pinctrl-0 = <&pwm1_pins_default>; + pinctrl-names = "default"; +}; + +&cpu_thermal { + cooling-maps { + map0 { + trip = <&cpu_alert>; + cooling-device = <&fan0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>; + }; + }; +}; + &ehci0 { status = "okay"; }; diff --git a/arch/loongarch/boot/dts/loongson-2k1000.dtsi b/arch/loongarch/boot/dts/loongson-2k1000.dtsi index 8dff2aa52417..1da3beb00f0e 100644 --- a/arch/loongarch/boot/dts/loongson-2k1000.dtsi +++ b/arch/loongarch/boot/dts/loongson-2k1000.dtsi @@ -68,7 +68,7 @@ }; thermal-zones { - cpu-thermal { + cpu_thermal: cpu-thermal { polling-delay-passive = <1000>; polling-delay = <5000>; thermal-sensors = <&tsensor 0>; @@ -322,6 +322,46 @@ status = "disabled"; }; + pwm@1fe22000 { + compatible = "loongson,ls2k1000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1fe22000 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm1: pwm@1fe22010 { + compatible = "loongson,ls2k1000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1fe22010 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1fe22020 { + compatible = "loongson,ls2k1000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1fe22020 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@1fe22030 { + compatible = "loongson,ls2k1000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x1fe22030 0x0 0x10>; + interrupt-parent = <&liointc0>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_APB_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + pmc: power-management@1fe27000 { compatible = "loongson,ls2k1000-pmc", "loongson,ls2k0500-pmc", "syscon"; reg = <0x0 0x1fe27000 0x0 0x58>; diff --git a/arch/loongarch/boot/dts/loongson-2k2000.dtsi b/arch/loongarch/boot/dts/loongson-2k2000.dtsi index b4ff55a33e90..9e0411f2754c 100644 --- a/arch/loongarch/boot/dts/loongson-2k2000.dtsi +++ b/arch/loongarch/boot/dts/loongson-2k2000.dtsi @@ -165,6 +165,66 @@ interrupt-parent = <&eiointc>; }; + pwm@100a0000 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0000 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <24 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@100a0100 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0100 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <25 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@100a0200 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0200 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <26 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@100a0300 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0300 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <27 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@100a0400 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0400 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <38 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + + pwm@100a0500 { + compatible = "loongson,ls2k2000-pwm", "loongson,ls7a-pwm"; + reg = <0x0 0x100a0500 0x0 0x10>; + interrupt-parent = <&pic>; + interrupts = <39 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&clk LOONGSON2_MISC_CLK>; + #pwm-cells = <3>; + status = "disabled"; + }; + rtc0: rtc@100d0100 { compatible = "loongson,ls2k2000-rtc", "loongson,ls7a-rtc"; reg = <0x0 0x100d0100 0x0 0x100>; diff --git a/arch/loongarch/boot/install.sh b/arch/loongarch/boot/install.sh new file mode 100755 index 000000000000..daac197d3315 --- /dev/null +++ b/arch/loongarch/boot/install.sh @@ -0,0 +1,56 @@ +#!/bin/sh +# +# This file is subject to the terms and conditions of the GNU General Public +# License. See the file "COPYING" in the main directory of this archive +# for more details. +# +# Copyright (C) 1995 by Linus Torvalds +# +# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin +# Adapted from code in arch/i386/boot/install.sh by Russell King +# +# "make install" script for the LoongArch Linux port +# +# Arguments: +# $1 - kernel version +# $2 - kernel image file +# $3 - kernel map file +# $4 - default install path (blank if root directory) + +set -e + +case "${2##*/}" in +vmlinux.elf) + echo "Installing uncompressed vmlinux.elf kernel" + base=vmlinux + ;; +vmlinux.efi) + echo "Installing uncompressed vmlinux.efi kernel" + base=vmlinux + ;; +vmlinuz.efi) + echo "Installing gzip/zstd compressed vmlinuz.efi kernel" + base=vmlinuz + ;; +*) + echo "Warning: Unexpected kernel type" + exit 1 + ;; +esac + +if [ -f $4/$base-$1 ]; then + mv $4/$base-$1 $4/$base-$1.old +fi +cat $2 > $4/$base-$1 + +# Install system map file +if [ -f $4/System.map-$1 ]; then + mv $4/System.map-$1 $4/System.map-$1.old +fi +cp $3 $4/System.map-$1 + +# Install kernel config file +if [ -f $4/config-$1 ]; then + mv $4/config-$1 $4/config-$1.old +fi +cp .config $4/config-$1 diff --git a/arch/loongarch/include/asm/acpi.h b/arch/loongarch/include/asm/acpi.h index 313f66f7913a..7376840fa9f7 100644 --- a/arch/loongarch/include/asm/acpi.h +++ b/arch/loongarch/include/asm/acpi.h @@ -33,7 +33,7 @@ static inline bool acpi_has_cpu_in_madt(void) return true; } -#define MAX_CORE_PIC 256 +#define MAX_CORE_PIC 2048 extern struct list_head acpi_wakeup_device_list; extern struct acpi_madt_core_pic acpi_core_pic[MAX_CORE_PIC]; diff --git a/arch/loongarch/include/asm/entry-common.h b/arch/loongarch/include/asm/entry-common.h index 0fe2a098ded9..099132980dc9 100644 --- a/arch/loongarch/include/asm/entry-common.h +++ b/arch/loongarch/include/asm/entry-common.h @@ -2,12 +2,6 @@ #ifndef ARCH_LOONGARCH_ENTRY_COMMON_H #define ARCH_LOONGARCH_ENTRY_COMMON_H -#include <linux/sched.h> -#include <linux/processor.h> - -static inline bool on_thread_stack(void) -{ - return !(((unsigned long)(current->stack) ^ current_stack_pointer) & ~(THREAD_SIZE - 1)); -} +#include <asm/stacktrace.h> /* For on_thread_stack() */ #endif diff --git a/arch/loongarch/include/asm/irqflags.h b/arch/loongarch/include/asm/irqflags.h index 319a8c616f1f..003172b8406b 100644 --- a/arch/loongarch/include/asm/irqflags.h +++ b/arch/loongarch/include/asm/irqflags.h @@ -14,40 +14,48 @@ static inline void arch_local_irq_enable(void) { u32 flags = CSR_CRMD_IE; + register u32 mask asm("t0") = CSR_CRMD_IE; + __asm__ __volatile__( "csrxchg %[val], %[mask], %[reg]\n\t" : [val] "+r" (flags) - : [mask] "r" (CSR_CRMD_IE), [reg] "i" (LOONGARCH_CSR_CRMD) + : [mask] "r" (mask), [reg] "i" (LOONGARCH_CSR_CRMD) : "memory"); } static inline void arch_local_irq_disable(void) { u32 flags = 0; + register u32 mask asm("t0") = CSR_CRMD_IE; + __asm__ __volatile__( "csrxchg %[val], %[mask], %[reg]\n\t" : [val] "+r" (flags) - : [mask] "r" (CSR_CRMD_IE), [reg] "i" (LOONGARCH_CSR_CRMD) + : [mask] "r" (mask), [reg] "i" (LOONGARCH_CSR_CRMD) : "memory"); } static inline unsigned long arch_local_irq_save(void) { u32 flags = 0; + register u32 mask asm("t0") = CSR_CRMD_IE; + __asm__ __volatile__( "csrxchg %[val], %[mask], %[reg]\n\t" : [val] "+r" (flags) - : [mask] "r" (CSR_CRMD_IE), [reg] "i" (LOONGARCH_CSR_CRMD) + : [mask] "r" (mask), [reg] "i" (LOONGARCH_CSR_CRMD) : "memory"); return flags; } static inline void arch_local_irq_restore(unsigned long flags) { + register u32 mask asm("t0") = CSR_CRMD_IE; + __asm__ __volatile__( "csrxchg %[val], %[mask], %[reg]\n\t" : [val] "+r" (flags) - : [mask] "r" (CSR_CRMD_IE), [reg] "i" (LOONGARCH_CSR_CRMD) + : [mask] "r" (mask), [reg] "i" (LOONGARCH_CSR_CRMD) : "memory"); } diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h index 52651aa0e583..d84dac88a584 100644 --- a/arch/loongarch/include/asm/loongarch.h +++ b/arch/loongarch/include/asm/loongarch.h @@ -411,8 +411,8 @@ /* Config CSR registers */ #define LOONGARCH_CSR_CPUID 0x20 /* CPU core id */ -#define CSR_CPUID_COREID_WIDTH 9 -#define CSR_CPUID_COREID _ULCAST_(0x1ff) +#define CSR_CPUID_COREID_WIDTH 11 +#define CSR_CPUID_COREID _ULCAST_(0x7ff) #define LOONGARCH_CSR_PRCFG1 0x21 /* Config1 */ #define CSR_CONF1_VSMAX_SHIFT 12 diff --git a/arch/loongarch/include/asm/numa.h b/arch/loongarch/include/asm/numa.h index b5f9de9f102e..bbf9f70bd25f 100644 --- a/arch/loongarch/include/asm/numa.h +++ b/arch/loongarch/include/asm/numa.h @@ -22,20 +22,6 @@ extern int numa_off; extern s16 __cpuid_to_node[CONFIG_NR_CPUS]; extern nodemask_t numa_nodes_parsed __initdata; -struct numa_memblk { - u64 start; - u64 end; - int nid; -}; - -#define NR_NODE_MEMBLKS (MAX_NUMNODES*2) -struct numa_meminfo { - int nr_blks; - struct numa_memblk blk[NR_NODE_MEMBLKS]; -}; - -extern int __init numa_add_memblk(int nodeid, u64 start, u64 end); - extern void __init early_numa_add_cpu(int cpuid, s16 node); extern void numa_add_cpu(unsigned int cpu); extern void numa_remove_cpu(unsigned int cpu); diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index a3f17914dbab..b30185302c07 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -301,7 +301,7 @@ static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset) #define __pmd_to_swp_entry(pmd) ((swp_entry_t) { pmd_val(pmd) }) #define __swp_entry_to_pmd(x) ((pmd_t) { (x).val | _PAGE_HUGE }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h index b87d1d5e5890..ad0bd234a0f1 100644 --- a/arch/loongarch/include/asm/smp.h +++ b/arch/loongarch/include/asm/smp.h @@ -25,6 +25,7 @@ extern int smp_num_siblings; extern int num_processors; extern int disabled_cpus; extern cpumask_t cpu_sibling_map[]; +extern cpumask_t cpu_llc_shared_map[]; extern cpumask_t cpu_core_map[]; extern cpumask_t cpu_foreign_map[]; diff --git a/arch/loongarch/include/asm/sparsemem.h b/arch/loongarch/include/asm/sparsemem.h index 8d4af6aff8a8..4501efac1a87 100644 --- a/arch/loongarch/include/asm/sparsemem.h +++ b/arch/loongarch/include/asm/sparsemem.h @@ -21,11 +21,6 @@ #define VMEMMAP_SIZE 0 /* 1, For FLATMEM; 2, For SPARSEMEM without VMEMMAP. */ #endif -#ifdef CONFIG_MEMORY_HOTPLUG -int memory_add_physaddr_to_nid(u64 addr); -#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid -#endif - #define INIT_MEMBLOCK_RESERVED_REGIONS (INIT_MEMBLOCK_REGIONS + NR_CPUS) #endif /* _LOONGARCH_SPARSEMEM_H */ diff --git a/arch/loongarch/include/asm/stackframe.h b/arch/loongarch/include/asm/stackframe.h index 66736837085b..3eda298702b1 100644 --- a/arch/loongarch/include/asm/stackframe.h +++ b/arch/loongarch/include/asm/stackframe.h @@ -57,6 +57,12 @@ jirl zero, \temp1, 0xc .endm + .macro STACKLEAK_ERASE +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + .endm + .macro BACKUP_T0T1 csrwr t0, EXCEPTION_KS0 csrwr t1, EXCEPTION_KS1 diff --git a/arch/loongarch/include/asm/stacktrace.h b/arch/loongarch/include/asm/stacktrace.h index fc8b64773794..5c8be156567c 100644 --- a/arch/loongarch/include/asm/stacktrace.h +++ b/arch/loongarch/include/asm/stacktrace.h @@ -31,6 +31,11 @@ bool in_irq_stack(unsigned long stack, struct stack_info *info); bool in_task_stack(unsigned long stack, struct task_struct *task, struct stack_info *info); int get_stack_info(unsigned long stack, struct task_struct *task, struct stack_info *info); +static __always_inline bool on_thread_stack(void) +{ + return !(((unsigned long)(current->stack) ^ current_stack_pointer) & ~(THREAD_SIZE - 1)); +} + #define STR_LONG_L __stringify(LONG_L) #define STR_LONG_S __stringify(LONG_S) #define STR_LONGSIZE __stringify(LONGSIZE) diff --git a/arch/loongarch/include/asm/topology.h b/arch/loongarch/include/asm/topology.h index 50273c9187d0..f06e7ff25bb7 100644 --- a/arch/loongarch/include/asm/topology.h +++ b/arch/loongarch/include/asm/topology.h @@ -19,17 +19,22 @@ extern int pcibus_to_node(struct pci_bus *); #define cpumask_of_pcibus(bus) (cpu_online_mask) -extern unsigned char node_distances[MAX_NUMNODES][MAX_NUMNODES]; - -void numa_set_distance(int from, int to, int distance); - -#define node_distance(from, to) (node_distances[(from)][(to)]) +int __node_distance(int from, int to); +#define node_distance(from, to) __node_distance(from, to) #else #define pcibus_to_node(bus) 0 #endif #ifdef CONFIG_SMP +/* + * Return cpus that shares the last level cache. + */ +static inline const struct cpumask *cpu_coregroup_mask(int cpu) +{ + return &cpu_llc_shared_map[cpu]; +} + #define topology_physical_package_id(cpu) (cpu_data[cpu].package) #define topology_core_id(cpu) (cpu_data[cpu].core) #define topology_core_cpumask(cpu) (&cpu_core_map[cpu]) diff --git a/arch/loongarch/include/asm/vdso/getrandom.h b/arch/loongarch/include/asm/vdso/getrandom.h index 48c43f55b039..a81724b69f29 100644 --- a/arch/loongarch/include/asm/vdso/getrandom.h +++ b/arch/loongarch/include/asm/vdso/getrandom.h @@ -20,7 +20,7 @@ static __always_inline ssize_t getrandom_syscall(void *_buffer, size_t _len, uns asm volatile( " syscall 0\n" - : "+r" (ret) + : "=r" (ret) : "r" (nr), "r" (buffer), "r" (len), "r" (flags) : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7", "$t8", "memory"); diff --git a/arch/loongarch/include/asm/vdso/gettimeofday.h b/arch/loongarch/include/asm/vdso/gettimeofday.h index 88cfcf133116..f15503e3336c 100644 --- a/arch/loongarch/include/asm/vdso/gettimeofday.h +++ b/arch/loongarch/include/asm/vdso/gettimeofday.h @@ -25,7 +25,7 @@ static __always_inline long gettimeofday_fallback( asm volatile( " syscall 0\n" - : "+r" (ret) + : "=r" (ret) : "r" (nr), "r" (tv), "r" (tz) : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7", "$t8", "memory"); @@ -44,7 +44,7 @@ static __always_inline long clock_gettime_fallback( asm volatile( " syscall 0\n" - : "+r" (ret) + : "=r" (ret) : "r" (nr), "r" (clkid), "r" (ts) : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7", "$t8", "memory"); @@ -63,7 +63,7 @@ static __always_inline int clock_getres_fallback( asm volatile( " syscall 0\n" - : "+r" (ret) + : "=r" (ret) : "r" (nr), "r" (clkid), "r" (ts) : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7", "$t8", "memory"); diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index f9dcaa60033d..6f5a4574a911 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -5,7 +5,7 @@ OBJECT_FILES_NON_STANDARD_head.o := y -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \ traps.o irq.o idle.o process.o dma.o mem.o reset.o switch.o \ diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c index 1120ac2824f6..a54cd6fd3796 100644 --- a/arch/loongarch/kernel/acpi.c +++ b/arch/loongarch/kernel/acpi.c @@ -244,22 +244,6 @@ fdt_earlycon: #ifdef CONFIG_ACPI_NUMA -static __init int setup_node(int pxm) -{ - return acpi_map_pxm_to_node(pxm); -} - -void __init numa_set_distance(int from, int to, int distance) -{ - if ((u8)distance != distance || (from == to && distance != LOCAL_DISTANCE)) { - pr_warn_once("Warning: invalid distance parameter, from=%d to=%d distance=%d\n", - from, to, distance); - return; - } - - node_distances[from][to] = distance; -} - /* Callback for Proximity Domain -> CPUID mapping */ void __init acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) @@ -280,7 +264,41 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) pxm |= (pa->proximity_domain_hi[1] << 16); pxm |= (pa->proximity_domain_hi[2] << 24); } - node = setup_node(pxm); + node = acpi_map_pxm_to_node(pxm); + if (node < 0) { + pr_err("SRAT: Too many proximity domains %x\n", pxm); + bad_srat(); + return; + } + + if (pa->apic_id >= CONFIG_NR_CPUS) { + pr_info("SRAT: PXM %u -> CPU 0x%02x -> Node %u skipped apicid that is too big\n", + pxm, pa->apic_id, node); + return; + } + + early_numa_add_cpu(pa->apic_id, node); + + set_cpuid_to_node(pa->apic_id, node); + node_set(node, numa_nodes_parsed); + pr_info("SRAT: PXM %u -> CPU 0x%02x -> Node %u\n", pxm, pa->apic_id, node); +} + +void __init +acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa) +{ + int pxm, node; + + if (srat_disabled()) + return; + if (pa->header.length < sizeof(struct acpi_srat_x2apic_cpu_affinity)) { + bad_srat(); + return; + } + if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0) + return; + pxm = pa->proximity_domain; + node = acpi_map_pxm_to_node(pxm); if (node < 0) { pr_err("SRAT: Too many proximity domains %x\n", pxm); bad_srat(); diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S index 2abc29e57381..47e1db9a1ce4 100644 --- a/arch/loongarch/kernel/entry.S +++ b/arch/loongarch/kernel/entry.S @@ -73,6 +73,7 @@ SYM_CODE_START(handle_syscall) move a0, sp bl do_syscall + STACKLEAK_ERASE RESTORE_ALL_AND_RET SYM_CODE_END(handle_syscall) _ASM_NOKPROBE(handle_syscall) @@ -81,6 +82,7 @@ SYM_CODE_START(ret_from_fork_asm) UNWIND_HINT_REGS move a1, sp bl ret_from_fork + STACKLEAK_ERASE RESTORE_STATIC RESTORE_SOME RESTORE_SP_AND_RET @@ -92,6 +94,7 @@ SYM_CODE_START(ret_from_kernel_thread_asm) move a2, s0 move a3, s1 bl ret_from_kernel_thread + STACKLEAK_ERASE RESTORE_STATIC RESTORE_SOME RESTORE_SP_AND_RET diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c index 30a72fd528c0..d6e73e8f9c0b 100644 --- a/arch/loongarch/kernel/numa.c +++ b/arch/loongarch/kernel/numa.c @@ -11,6 +11,7 @@ #include <linux/mmzone.h> #include <linux/export.h> #include <linux/nodemask.h> +#include <linux/numa_memblks.h> #include <linux/swap.h> #include <linux/memblock.h> #include <linux/pfn.h> @@ -27,10 +28,6 @@ #include <asm/time.h> int numa_off; -unsigned char node_distances[MAX_NUMNODES][MAX_NUMNODES]; -EXPORT_SYMBOL(node_distances); - -static struct numa_meminfo numa_meminfo; cpumask_t cpus_on_node[MAX_NUMNODES]; cpumask_t phys_cpus_on_node[MAX_NUMNODES]; EXPORT_SYMBOL(cpus_on_node); @@ -43,8 +40,6 @@ s16 __cpuid_to_node[CONFIG_NR_CPUS] = { }; EXPORT_SYMBOL(__cpuid_to_node); -nodemask_t numa_nodes_parsed __initdata; - #ifdef CONFIG_HAVE_SETUP_PER_CPU_AREA unsigned long __per_cpu_offset[NR_CPUS] __read_mostly; EXPORT_SYMBOL(__per_cpu_offset); @@ -145,48 +140,6 @@ void numa_remove_cpu(unsigned int cpu) cpumask_clear_cpu(cpu, &cpus_on_node[nid]); } -static int __init numa_add_memblk_to(int nid, u64 start, u64 end, - struct numa_meminfo *mi) -{ - /* ignore zero length blks */ - if (start == end) - return 0; - - /* whine about and ignore invalid blks */ - if (start > end || nid < 0 || nid >= MAX_NUMNODES) { - pr_warn("NUMA: Warning: invalid memblk node %d [mem %#010Lx-%#010Lx]\n", - nid, start, end - 1); - return 0; - } - - if (mi->nr_blks >= NR_NODE_MEMBLKS) { - pr_err("NUMA: too many memblk ranges\n"); - return -EINVAL; - } - - mi->blk[mi->nr_blks].start = PFN_ALIGN(start); - mi->blk[mi->nr_blks].end = PFN_ALIGN(end - PAGE_SIZE + 1); - mi->blk[mi->nr_blks].nid = nid; - mi->nr_blks++; - return 0; -} - -/** - * numa_add_memblk - Add one numa_memblk to numa_meminfo - * @nid: NUMA node ID of the new memblk - * @start: Start address of the new memblk - * @end: End address of the new memblk - * - * Add a new memblk to the default numa_meminfo. - * - * RETURNS: - * 0 on success, -errno on failure. - */ -int __init numa_add_memblk(int nid, u64 start, u64 end) -{ - return numa_add_memblk_to(nid, start, end, &numa_meminfo); -} - static void __init node_mem_init(unsigned int node) { unsigned long start_pfn, end_pfn; @@ -205,18 +158,6 @@ static void __init node_mem_init(unsigned int node) #ifdef CONFIG_ACPI_NUMA -static void __init add_node_intersection(u32 node, u64 start, u64 size, u32 type) -{ - static unsigned long num_physpages; - - num_physpages += (size >> PAGE_SHIFT); - pr_info("Node%d: mem_type:%d, mem_start:0x%llx, mem_size:0x%llx Bytes\n", - node, type, start, size); - pr_info(" start_pfn:0x%llx, end_pfn:0x%llx, num_physpages:0x%lx\n", - start >> PAGE_SHIFT, (start + size) >> PAGE_SHIFT, num_physpages); - memblock_set_node(start, size, &memblock.memory, node); -} - /* * add_numamem_region * @@ -228,28 +169,21 @@ static void __init add_node_intersection(u32 node, u64 start, u64 size, u32 type */ static void __init add_numamem_region(u64 start, u64 end, u32 type) { - u32 i; - u64 ofs = start; + u32 node = pa_to_nid(start); + u64 size = end - start; + static unsigned long num_physpages; if (start >= end) { pr_debug("Invalid region: %016llx-%016llx\n", start, end); return; } - for (i = 0; i < numa_meminfo.nr_blks; i++) { - struct numa_memblk *mb = &numa_meminfo.blk[i]; - - if (ofs > mb->end) - continue; - - if (end > mb->end) { - add_node_intersection(mb->nid, ofs, mb->end - ofs, type); - ofs = mb->end; - } else { - add_node_intersection(mb->nid, ofs, end - ofs, type); - break; - } - } + num_physpages += (size >> PAGE_SHIFT); + pr_info("Node%d: mem_type:%d, mem_start:0x%llx, mem_size:0x%llx Bytes\n", + node, type, start, size); + pr_info(" start_pfn:0x%llx, end_pfn:0x%llx, num_physpages:0x%lx\n", + start >> PAGE_SHIFT, end >> PAGE_SHIFT, num_physpages); + memblock_set_node(start, size, &memblock.memory, node); } static void __init init_node_memblock(void) @@ -291,24 +225,6 @@ static void __init init_node_memblock(void) } } -static void __init numa_default_distance(void) -{ - int row, col; - - for (row = 0; row < MAX_NUMNODES; row++) - for (col = 0; col < MAX_NUMNODES; col++) { - if (col == row) - node_distances[row][col] = LOCAL_DISTANCE; - else - /* We assume that one node per package here! - * - * A SLIT should be used for multiple nodes - * per package to override default setting. - */ - node_distances[row][col] = REMOTE_DISTANCE; - } -} - /* * fake_numa_init() - For Non-ACPI systems * Return: 0 on success, -errno on failure. @@ -333,11 +249,11 @@ int __init init_numa_memory(void) for (i = 0; i < NR_CPUS; i++) set_cpuid_to_node(i, NUMA_NO_NODE); - numa_default_distance(); + numa_reset_distance(); nodes_clear(numa_nodes_parsed); nodes_clear(node_possible_map); nodes_clear(node_online_map); - memset(&numa_meminfo, 0, sizeof(numa_meminfo)); + WARN_ON(memblock_clear_hotplug(0, PHYS_ADDR_MAX)); /* Parse SRAT and SLIT if provided by firmware. */ ret = acpi_disabled ? fake_numa_init() : acpi_numa_init(); diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index 4b24589c0b56..46036d98da75 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -46,6 +46,10 @@ EXPORT_SYMBOL(__cpu_logical_map); cpumask_t cpu_sibling_map[NR_CPUS] __read_mostly; EXPORT_SYMBOL(cpu_sibling_map); +/* Representing the last level cache shared map of each logical CPU */ +cpumask_t cpu_llc_shared_map[NR_CPUS] __read_mostly; +EXPORT_SYMBOL(cpu_llc_shared_map); + /* Representing the core map of multi-core chips of each logical CPU */ cpumask_t cpu_core_map[NR_CPUS] __read_mostly; EXPORT_SYMBOL(cpu_core_map); @@ -63,6 +67,9 @@ EXPORT_SYMBOL(cpu_foreign_map); /* representing cpus for which sibling maps can be computed */ static cpumask_t cpu_sibling_setup_map; +/* representing cpus for which llc shared maps can be computed */ +static cpumask_t cpu_llc_shared_setup_map; + /* representing cpus for which core maps can be computed */ static cpumask_t cpu_core_setup_map; @@ -102,6 +109,34 @@ static inline void set_cpu_core_map(int cpu) } } +static inline void set_cpu_llc_shared_map(int cpu) +{ + int i; + + cpumask_set_cpu(cpu, &cpu_llc_shared_setup_map); + + for_each_cpu(i, &cpu_llc_shared_setup_map) { + if (cpu_to_node(cpu) == cpu_to_node(i)) { + cpumask_set_cpu(i, &cpu_llc_shared_map[cpu]); + cpumask_set_cpu(cpu, &cpu_llc_shared_map[i]); + } + } +} + +static inline void clear_cpu_llc_shared_map(int cpu) +{ + int i; + + for_each_cpu(i, &cpu_llc_shared_setup_map) { + if (cpu_to_node(cpu) == cpu_to_node(i)) { + cpumask_clear_cpu(i, &cpu_llc_shared_map[cpu]); + cpumask_clear_cpu(cpu, &cpu_llc_shared_map[i]); + } + } + + cpumask_clear_cpu(cpu, &cpu_llc_shared_setup_map); +} + static inline void set_cpu_sibling_map(int cpu) { int i; @@ -406,6 +441,7 @@ int loongson_cpu_disable(void) #endif set_cpu_online(cpu, false); clear_cpu_sibling_map(cpu); + clear_cpu_llc_shared_map(cpu); calculate_cpu_foreign_map(); local_irq_save(flags); irq_migrate_all_off_this_cpu(); @@ -572,6 +608,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus) current_thread_info()->cpu = 0; loongson_prepare_cpus(max_cpus); set_cpu_sibling_map(0); + set_cpu_llc_shared_map(0); set_cpu_core_map(0); calculate_cpu_foreign_map(); #ifndef CONFIG_HOTPLUG_CPU @@ -613,6 +650,7 @@ asmlinkage void start_secondary(void) loongson_init_secondary(); set_cpu_sibling_map(cpu); + set_cpu_llc_shared_map(cpu); set_cpu_core_map(cpu); notify_cpu_starting(cpu); diff --git a/arch/loongarch/kernel/vdso.c b/arch/loongarch/kernel/vdso.c index 10cf1608c7b3..7b888d9085a0 100644 --- a/arch/loongarch/kernel/vdso.c +++ b/arch/loongarch/kernel/vdso.c @@ -105,7 +105,9 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) vdso_addr = data_addr + VVAR_SIZE; vma = _install_special_mapping(mm, vdso_addr, info->size, - VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC, + VM_READ | VM_EXEC | + VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC | + VM_SEALED_SYSMAP, &info->code_mapping); if (IS_ERR(vma)) { ret = PTR_ERR(vma); diff --git a/arch/loongarch/mm/hugetlbpage.c b/arch/loongarch/mm/hugetlbpage.c index cea84d7f2b91..02dad4624fe3 100644 --- a/arch/loongarch/mm/hugetlbpage.c +++ b/arch/loongarch/mm/hugetlbpage.c @@ -47,7 +47,8 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, pmd = pmd_offset(pud, addr); } } - return pmd_none(pmdp_get(pmd)) ? NULL : (pte_t *) pmd; + + return (!pmd || pmd_none(pmdp_get(pmd))) ? NULL : (pte_t *) pmd; } uint64_t pmd_to_entrylo(unsigned long pmd_val) diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index 06f11d9e4ec1..c3e4586a7975 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -106,14 +106,6 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) page += vmem_altmap_offset(altmap); __remove_pages(start_pfn, nr_pages, altmap); } - -#ifdef CONFIG_NUMA -int memory_add_physaddr_to_nid(u64 start) -{ - return pa_to_nid(start); -} -EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid); -#endif #endif #ifdef CONFIG_SPARSEMEM_VMEMMAP diff --git a/arch/loongarch/pci/acpi.c b/arch/loongarch/pci/acpi.c index 1da4dc46df43..50c9016641a4 100644 --- a/arch/loongarch/pci/acpi.c +++ b/arch/loongarch/pci/acpi.c @@ -194,6 +194,7 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root) { struct pci_bus *bus; struct pci_root_info *info; + struct pci_host_bridge *host; struct acpi_pci_root_ops *root_ops; int domain = root->segment; int busnum = root->secondary.start; @@ -237,8 +238,17 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root) return NULL; } - pci_bus_size_bridges(bus); - pci_bus_assign_resources(bus); + /* If we must preserve the resource configuration, claim now */ + host = pci_find_host_bridge(bus); + if (host->preserve_config) + pci_bus_claim_resources(bus); + + /* + * Assign whatever was left unassigned. If we didn't claim above, + * this will reassign everything. + */ + pci_assign_unassigned_root_bus_resources(bus); + list_for_each_entry(child, &bus->children, node) pcie_bus_configure_settings(child); } diff --git a/arch/m68k/include/asm/mcf_pgtable.h b/arch/m68k/include/asm/mcf_pgtable.h index f5c596b211d4..d79fef609194 100644 --- a/arch/m68k/include/asm/mcf_pgtable.h +++ b/arch/m68k/include/asm/mcf_pgtable.h @@ -268,7 +268,7 @@ extern pgd_t kernel_pg_dir[PTRS_PER_PGD]; #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) (__pte((x).val)) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/m68k/include/asm/motorola_pgtable.h b/arch/m68k/include/asm/motorola_pgtable.h index 040ac3bad713..14fee64d3e60 100644 --- a/arch/m68k/include/asm/motorola_pgtable.h +++ b/arch/m68k/include/asm/motorola_pgtable.h @@ -185,7 +185,7 @@ extern pgd_t kernel_pg_dir[128]; #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/m68k/include/asm/sun3_pgtable.h b/arch/m68k/include/asm/sun3_pgtable.h index 73745dc0ec0e..858cbe936f5b 100644 --- a/arch/m68k/include/asm/sun3_pgtable.h +++ b/arch/m68k/include/asm/sun3_pgtable.h @@ -169,7 +169,7 @@ extern pgd_t kernel_pg_dir[PTRS_PER_PGD]; #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/m68k/kernel/Makefile b/arch/m68k/kernel/Makefile index 6c732ed3998b..57c1b3e8d60e 100644 --- a/arch/m68k/kernel/Makefile +++ b/arch/m68k/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the linux kernel. # -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds obj-$(CONFIG_MMU_MOTOROLA) := head.o obj-$(CONFIG_SUN3) := sun3-head.o diff --git a/arch/microblaze/include/asm/pgtable.h b/arch/microblaze/include/asm/pgtable.h index b1bb2c65dd04..bae1abfa6f6b 100644 --- a/arch/microblaze/include/asm/pgtable.h +++ b/arch/microblaze/include/asm/pgtable.h @@ -398,7 +398,7 @@ extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) >> 2 }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val << 2 }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/microblaze/kernel/Makefile b/arch/microblaze/kernel/Makefile index 85c4d29ef43e..241e466e7333 100644 --- a/arch/microblaze/kernel/Makefile +++ b/arch/microblaze/kernel/Makefile @@ -11,7 +11,7 @@ CFLAGS_REMOVE_ftrace.o = -pg CFLAGS_REMOVE_process.o = -pg endif -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y += head.o dma.o exceptions.o \ hw_exception_handler.o irq.o \ diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h index 4852b005a72d..ae73ecf4c41a 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -534,7 +534,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) #endif #if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte.pte_low & _PAGE_SWP_EXCLUSIVE; } @@ -551,7 +551,7 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte) return pte; } #else -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile index ecf3278a32f7..95a1e674fd67 100644 --- a/arch/mips/kernel/Makefile +++ b/arch/mips/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the Linux/MIPS kernel. # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y += head.o branch.o cmpxchg.o elf.o entry.o genex.o idle.o irq.o \ process.o prom.o ptrace.o reset.o setup.o signal.o \ diff --git a/arch/nios2/include/asm/pgtable.h b/arch/nios2/include/asm/pgtable.h index e98578e27e26..844dce55569f 100644 --- a/arch/nios2/include/asm/pgtable.h +++ b/arch/nios2/include/asm/pgtable.h @@ -259,7 +259,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) #define __swp_entry_to_pte(swp) ((pte_t) { (swp).val }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/nios2/kernel/Makefile b/arch/nios2/kernel/Makefile index 78a913181fa1..4dce965a7b73 100644 --- a/arch/nios2/kernel/Makefile +++ b/arch/nios2/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the nios2 linux kernel. # -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds obj-y += head.o obj-y += cpuinfo.o diff --git a/arch/openrisc/include/asm/pgtable.h b/arch/openrisc/include/asm/pgtable.h index 71bfb8c8c482..5bd6463bd514 100644 --- a/arch/openrisc/include/asm/pgtable.h +++ b/arch/openrisc/include/asm/pgtable.h @@ -411,7 +411,7 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf, #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/openrisc/kernel/Makefile b/arch/openrisc/kernel/Makefile index e4c7d9bdd598..58e6a1b525b7 100644 --- a/arch/openrisc/kernel/Makefile +++ b/arch/openrisc/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the linux kernel. # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y := head.o setup.o or32_ksyms.o process.o dma.o \ traps.o time.o irq.o entry.o ptrace.o signal.o \ diff --git a/arch/parisc/include/asm/pgtable.h b/arch/parisc/include/asm/pgtable.h index 80f5e2a28413..1a86a4370b29 100644 --- a/arch/parisc/include/asm/pgtable.h +++ b/arch/parisc/include/asm/pgtable.h @@ -425,7 +425,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/parisc/kernel/Makefile b/arch/parisc/kernel/Makefile index 5ab0467be70a..d5055ba33722 100644 --- a/arch/parisc/kernel/Makefile +++ b/arch/parisc/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for arch/parisc/kernel # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y := head.o cache.o pacache.o setup.o pdt.o traps.o time.o irq.o \ syscall.o entry.o sys_parisc.o firmware.o \ diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h b/arch/powerpc/include/asm/book3s/32/pgtable.h index 42c3af90d1f0..92d21c6faf1e 100644 --- a/arch/powerpc/include/asm/book3s/32/pgtable.h +++ b/arch/powerpc/include/asm/book3s/32/pgtable.h @@ -365,7 +365,7 @@ static inline void __ptep_set_access_flags(struct vm_area_struct *vma, #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) >> 3 }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val << 3 }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 6ed93e290c2f..a2ddcbb3fcb9 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -693,7 +693,7 @@ static inline pte_t pte_swp_mkexclusive(pte_t pte) return __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_SWP_EXCLUSIVE)); } -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return !!(pte_raw(pte) & cpu_to_be64(_PAGE_SWP_EXCLUSIVE)); } diff --git a/arch/powerpc/include/asm/nohash/pgtable.h b/arch/powerpc/include/asm/nohash/pgtable.h index 8d1f0b7062eb..7d6b9e5b286e 100644 --- a/arch/powerpc/include/asm/nohash/pgtable.h +++ b/arch/powerpc/include/asm/nohash/pgtable.h @@ -286,7 +286,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot)); } -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile index 9d1ab3971694..fb2b95267022 100644 --- a/arch/powerpc/kernel/Makefile +++ b/arch/powerpc/kernel/Makefile @@ -126,7 +126,7 @@ obj-$(CONFIG_PPC_BOOK3S_32) += head_book3s_32.o obj-$(CONFIG_44x) += head_44x.o obj-$(CONFIG_PPC_8xx) += head_8xx.o obj-$(CONFIG_PPC_85xx) += head_85xx.o -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds obj-$(CONFIG_RELOCATABLE) += reloc_$(BITS).o diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 791d1942a058..3401b96be475 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -628,7 +628,7 @@ static void arm_next_watchdog(struct kvm_vcpu *vcpu) static void kvmppc_watchdog_func(struct timer_list *t) { - struct kvm_vcpu *vcpu = from_timer(vcpu, t, arch.wdt_timer); + struct kvm_vcpu *vcpu = timer_container_of(vcpu, t, arch.wdt_timer); u32 tsr, new_tsr; int final; diff --git a/arch/powerpc/platforms/book3s/vas-api.c b/arch/powerpc/platforms/book3s/vas-api.c index 0b6365d85d11..dc6f75d3ac6e 100644 --- a/arch/powerpc/platforms/book3s/vas-api.c +++ b/arch/powerpc/platforms/book3s/vas-api.c @@ -521,6 +521,15 @@ static int coproc_mmap(struct file *fp, struct vm_area_struct *vma) return -EINVAL; } + /* + * Map complete page to the paste address. So the user + * space should pass 0ULL to the offset parameter. + */ + if (vma->vm_pgoff) { + pr_debug("Page offset unsupported to map paste address\n"); + return -EINVAL; + } + /* Ensure instance has an open send window */ if (!txwin) { pr_err("No send window open?\n"); diff --git a/arch/powerpc/platforms/powermac/low_i2c.c b/arch/powerpc/platforms/powermac/low_i2c.c index a0ae58636e10..02474e27df9b 100644 --- a/arch/powerpc/platforms/powermac/low_i2c.c +++ b/arch/powerpc/platforms/powermac/low_i2c.c @@ -359,7 +359,8 @@ static irqreturn_t kw_i2c_irq(int irq, void *dev_id) static void kw_i2c_timeout(struct timer_list *t) { - struct pmac_i2c_host_kw *host = from_timer(host, t, timeout_timer); + struct pmac_i2c_host_kw *host = timer_container_of(host, t, + timeout_timer); unsigned long flags; spin_lock_irqsave(&host->lock, flags); diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c index 4ac9808e55a4..2ea30b343354 100644 --- a/arch/powerpc/platforms/powernv/memtrace.c +++ b/arch/powerpc/platforms/powernv/memtrace.c @@ -48,11 +48,15 @@ static ssize_t memtrace_read(struct file *filp, char __user *ubuf, static int memtrace_mmap(struct file *filp, struct vm_area_struct *vma) { struct memtrace_entry *ent = filp->private_data; + unsigned long ent_nrpages = ent->size >> PAGE_SHIFT; + unsigned long vma_nrpages = vma_pages(vma); - if (ent->size < vma->vm_end - vma->vm_start) + /* The requested page offset should be within object's page count */ + if (vma->vm_pgoff >= ent_nrpages) return -EINVAL; - if (vma->vm_pgoff << PAGE_SHIFT >= ent->size) + /* The requested mapping range should remain within the bounds */ + if (vma_nrpages > ent_nrpages - vma->vm_pgoff) return -EINVAL; vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index bbec87b79309..36061f4732b7 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -70,6 +70,7 @@ config RISCV # LLD >= 14: https://github.com/llvm/llvm-project/issues/50505 select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000 select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000 + select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS if 64BIT && MMU select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU select ARCH_SUPPORTS_PER_VMA_LOCK if MMU select ARCH_SUPPORTS_RT @@ -99,6 +100,7 @@ config RISCV select EDAC_SUPPORT select FRAME_POINTER if PERF_EVENTS || (FUNCTION_TRACER && !DYNAMIC_FTRACE) select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE + select FUNCTION_ALIGNMENT_8B if DYNAMIC_FTRACE_WITH_CALL_OPS select GENERIC_ARCH_TOPOLOGY select GENERIC_ATOMIC64 if !64BIT select GENERIC_CLOCKEVENTS_BROADCAST if SMP @@ -143,6 +145,7 @@ config RISCV select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE if 64BIT && MMU + select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if 64BIT && MMU select HAVE_ARCH_USERFAULTFD_MINOR if 64BIT && USERFAULTFD select HAVE_ARCH_VMAP_STACK if MMU && 64BIT select HAVE_ASM_MODVERSIONS @@ -150,13 +153,15 @@ config RISCV select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS if MMU select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && (CLANG_SUPPORTS_DYNAMIC_FTRACE || GCC_SUPPORTS_DYNAMIC_FTRACE) - select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + select FUNCTION_ALIGNMENT_4B if HAVE_DYNAMIC_FTRACE && RISCV_ISA_C + select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS if HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS + select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS if (DYNAMIC_FTRACE_WITH_ARGS && !CFI_CLANG) select HAVE_DYNAMIC_FTRACE_WITH_ARGS if HAVE_DYNAMIC_FTRACE select HAVE_FTRACE_GRAPH_FUNC select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if HAVE_DYNAMIC_FTRACE_WITH_ARGS select HAVE_FUNCTION_GRAPH_FREGS - select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION + select HAVE_FUNCTION_TRACER if !XIP_KERNEL select HAVE_EBPF_JIT if MMU select HAVE_GUP_FAST if MMU select HAVE_FUNCTION_ARG_ACCESS_API @@ -218,6 +223,7 @@ config RISCV select THREAD_INFO_IN_TASK select TRACE_IRQFLAGS_SUPPORT select UACCESS_MEMCPY if !MMU + select VDSO_GETRANDOM if HAVE_GENERIC_VDSO select USER_STACKTRACE_SUPPORT select ZONE_DMA32 if 64BIT @@ -236,6 +242,7 @@ config CLANG_SUPPORTS_DYNAMIC_FTRACE config GCC_SUPPORTS_DYNAMIC_FTRACE def_bool CC_IS_GCC depends on $(cc-option,-fpatchable-function-entry=8) + depends on CC_HAS_MIN_FUNCTION_ALIGNMENT || !RISCV_ISA_C config HAVE_SHADOW_CALL_STACK def_bool $(cc-option,-fsanitize=shadow-call-stack) @@ -664,12 +671,12 @@ config RISCV_ISA_V_PREEMPTIVE default y help Usually, in-kernel SIMD routines are run with preemption disabled. - Functions which envoke long running SIMD thus must yield core's + Functions which invoke long running SIMD thus must yield the core's vector unit to prevent blocking other tasks for too long. - This config allows kernel to run SIMD without explicitly disable - preemption. Enabling this config will result in higher memory - consumption due to the allocation of per-task's kernel Vector context. + This config allows the kernel to run SIMD without explicitly disabling + preemption. Enabling this config will result in higher memory consumption + due to the allocation of per-task's kernel Vector context. config RISCV_ISA_ZAWRS bool "Zawrs extension support for more efficient busy waiting" @@ -842,6 +849,21 @@ config RISCV_ISA_ZICBOZ If you don't know what to do here, say Y. +config RISCV_ISA_ZICBOP + bool "Zicbop extension support for cache block prefetch" + depends on MMU + depends on RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZICBOP + extension (Cache Block Prefetch Operations) and enable its + usage. + + The Zicbop extension can be used to prefetch cache blocks for + read/write fetch. + + If you don't know what to do here, say Y. + config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI def_bool y # https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc @@ -1171,8 +1193,8 @@ config CMDLINE_FALLBACK config CMDLINE_EXTEND bool "Extend bootloader kernel arguments" help - The command-line arguments provided during boot will be - appended to the built-in command line. This is useful in + The built-in command line will be appended to the command- + line arguments provided during boot. This is useful in cases where the provided arguments are insufficient and you don't want to or cannot modify them. diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor index b096548fe0ff..e14f26368963 100644 --- a/arch/riscv/Kconfig.vendor +++ b/arch/riscv/Kconfig.vendor @@ -16,6 +16,19 @@ config RISCV_ISA_VENDOR_EXT_ANDES If you don't know what to do here, say Y. endmenu +menu "SiFive" +config RISCV_ISA_VENDOR_EXT_SIFIVE + bool "SiFive vendor extension support" + select RISCV_ISA_VENDOR_EXT + default y + help + Say N here if you want to disable all SiFive vendor extension + support. This will cause any SiFive vendor extensions that are + requested by hardware probing to be ignored. + + If you don't know what to do here, say Y. +endmenu + menu "T-Head" config RISCV_ISA_VENDOR_EXT_THEAD bool "T-Head vendor extension support" diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 539d2aef5cab..df57654a615e 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -15,9 +15,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE),y) LDFLAGS_vmlinux += --no-relax KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY ifeq ($(CONFIG_RISCV_ISA_C),y) - CC_FLAGS_FTRACE := -fpatchable-function-entry=4 + CC_FLAGS_FTRACE := -fpatchable-function-entry=8,4 else - CC_FLAGS_FTRACE := -fpatchable-function-entry=2 + CC_FLAGS_FTRACE := -fpatchable-function-entry=4,2 endif endif diff --git a/arch/riscv/configs/defconfig b/arch/riscv/configs/defconfig index eea825ee58e1..fe8bd8afb418 100644 --- a/arch/riscv/configs/defconfig +++ b/arch/riscv/configs/defconfig @@ -18,12 +18,9 @@ CONFIG_CGROUP_DEVICE=y CONFIG_CGROUP_CPUACCT=y CONFIG_CGROUP_PERF=y CONFIG_CGROUP_BPF=y -CONFIG_NAMESPACES=y CONFIG_USER_NS=y CONFIG_CHECKPOINT_RESTORE=y CONFIG_BLK_DEV_INITRD=y -CONFIG_EXPERT=y -# CONFIG_SYSFS_SYSCALL is not set CONFIG_PROFILING=y CONFIG_ARCH_MICROCHIP=y CONFIG_ARCH_SIFIVE=y @@ -182,6 +179,7 @@ CONFIG_REGULATOR_FIXED_VOLTAGE=y CONFIG_REGULATOR_AXP20X=y CONFIG_REGULATOR_GPIO=y CONFIG_MEDIA_SUPPORT=m +CONFIG_MEDIA_PLATFORM_SUPPORT=y CONFIG_VIDEO_CADENCE_CSI2RX=m CONFIG_DRM=m CONFIG_DRM_RADEON=m @@ -297,25 +295,7 @@ CONFIG_DEFAULT_SECURITY_DAC=y CONFIG_CRYPTO_USER_API_HASH=y CONFIG_CRYPTO_DEV_VIRTIO=y CONFIG_PRINTK_TIME=y +CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_FS=y -CONFIG_DEBUG_PAGEALLOC=y -CONFIG_SCHED_STACK_END_CHECK=y -CONFIG_DEBUG_VM=y -CONFIG_DEBUG_VM_PGFLAGS=y -CONFIG_DEBUG_MEMORY_INIT=y -CONFIG_DEBUG_PER_CPU_MAPS=y -CONFIG_SOFTLOCKUP_DETECTOR=y -CONFIG_WQ_WATCHDOG=y -CONFIG_DEBUG_RT_MUTEXES=y -CONFIG_DEBUG_SPINLOCK=y -CONFIG_DEBUG_MUTEXES=y -CONFIG_DEBUG_RWSEMS=y -CONFIG_DEBUG_ATOMIC_SLEEP=y -CONFIG_DEBUG_LIST=y -CONFIG_DEBUG_PLIST=y -CONFIG_DEBUG_SG=y -# CONFIG_RCU_TRACE is not set -CONFIG_RCU_EQS_DEBUG=y -# CONFIG_FTRACE is not set # CONFIG_RUNTIME_TESTING_MENU is not set CONFIG_MEMTEST=y diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h index bfc8ea5f9319..a9988bf21ec8 100644 --- a/arch/riscv/include/asm/asm-prototypes.h +++ b/arch/riscv/include/asm/asm-prototypes.h @@ -12,7 +12,7 @@ long long __ashlti3(long long a, int b); #ifdef CONFIG_RISCV_ISA_V #ifdef CONFIG_MMU -asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n); +asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n, bool enable_sum); #endif /* CONFIG_MMU */ void xor_regs_2_(unsigned long bytes, unsigned long *__restrict p1, diff --git a/arch/riscv/include/asm/barrier.h b/arch/riscv/include/asm/barrier.h index e1d9bf1deca6..b8c5726d86ac 100644 --- a/arch/riscv/include/asm/barrier.h +++ b/arch/riscv/include/asm/barrier.h @@ -14,11 +14,6 @@ #include <asm/cmpxchg.h> #include <asm/fence.h> -#define nop() __asm__ __volatile__ ("nop") -#define __nops(n) ".rept " #n "\nnop\n.endr\n" -#define nops(n) __asm__ __volatile__ (__nops(n)) - - /* These barriers need to enforce ordering on both devices or memory. */ #define __mb() RISCV_FENCE(iorw, iorw) #define __rmb() RISCV_FENCE(ir, ir) diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index b59ffeb668d6..6086b38d5427 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -85,6 +85,7 @@ static inline void flush_icache_range(unsigned long start, unsigned long end) extern unsigned int riscv_cbom_block_size; extern unsigned int riscv_cboz_block_size; +extern unsigned int riscv_cbop_block_size; void riscv_init_cbo_blocksizes(void); #ifdef CONFIG_RISCV_DMA_NONCOHERENT diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 2ec119eb147b..0b749e710216 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -13,6 +13,7 @@ #include <asm/hwcap.h> #include <asm/insn-def.h> #include <asm/cpufeature-macros.h> +#include <asm/processor.h> #define __arch_xchg_masked(sc_sfx, swap_sfx, prepend, sc_append, \ swap_append, r, p, n) \ @@ -37,6 +38,7 @@ \ __asm__ __volatile__ ( \ prepend \ + PREFETCHW_ASM(%5) \ "0: lr.w %0, %2\n" \ " and %1, %0, %z4\n" \ " or %1, %1, %z3\n" \ @@ -44,7 +46,7 @@ " bnez %1, 0b\n" \ sc_append \ : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ - : "rJ" (__newx), "rJ" (~__mask) \ + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \ : "memory"); \ \ r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index f56b409361fb..fbd0e4306c93 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -67,11 +67,11 @@ void __init riscv_user_isa_enable(void); _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts), _validate) bool __init check_unaligned_access_emulated_all_cpus(void); +void unaligned_access_init(void); +int cpu_online_unaligned_access_init(unsigned int cpu); #if defined(CONFIG_RISCV_SCALAR_MISALIGNED) -void check_unaligned_access_emulated(struct work_struct *work __always_unused); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); -DECLARE_PER_CPU(long, misaligned_access_speed); #else static inline bool unaligned_ctl_available(void) { @@ -79,6 +79,16 @@ static inline bool unaligned_ctl_available(void) } #endif +#if defined(CONFIG_RISCV_MISALIGNED) +DECLARE_PER_CPU(long, misaligned_access_speed); +bool misaligned_traps_can_delegate(void); +#else +static inline bool misaligned_traps_can_delegate(void) +{ + return false; +} +#endif + bool __init check_vector_unaligned_access_emulated_all_cpus(void); #if defined(CONFIG_RISCV_VECTOR_MISALIGNED) void check_vector_unaligned_access_emulated(struct work_struct *work __always_unused); diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index d627f63ee289..22ebea3c2b26 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -20,10 +20,9 @@ extern void *return_address(unsigned int level); #define ftrace_return_address(n) return_address(n) void _mcount(void); -static inline unsigned long ftrace_call_adjust(unsigned long addr) -{ - return addr; -} +unsigned long ftrace_call_adjust(unsigned long addr); +unsigned long arch_ftrace_get_symaddr(unsigned long fentry_ip); +#define ftrace_get_symaddr(fentry_ip) arch_ftrace_get_symaddr(fentry_ip) /* * Let's do like x86/arm64 and ignore the compat syscalls. @@ -57,12 +56,21 @@ struct dyn_arch_ftrace { * 2) jalr: setting low-12 offset to ra, jump to ra, and set ra to * return address (original pc + 4) * + * The first 2 instructions for each tracable function is compiled to 2 nop + * instructions. Then, the kernel initializes the first instruction to auipc at + * boot time (<ftrace disable>). The second instruction is patched to jalr to + * start the trace. + * + *<Image>: + * 0: nop + * 4: nop + * *<ftrace enable>: - * 0: auipc t0/ra, 0x? - * 4: jalr t0/ra, ?(t0/ra) + * 0: auipc t0, 0x? + * 4: jalr t0, ?(t0) * *<ftrace disable>: - * 0: nop + * 0: auipc t0, 0x? * 4: nop * * Dynamic ftrace generates probes to call sites, so we must deal with @@ -75,10 +83,9 @@ struct dyn_arch_ftrace { #define AUIPC_OFFSET_MASK (0xfffff000) #define AUIPC_PAD (0x00001000) #define JALR_SHIFT 20 -#define JALR_RA (0x000080e7) -#define AUIPC_RA (0x00000097) #define JALR_T0 (0x000282e7) #define AUIPC_T0 (0x00000297) +#define JALR_RANGE (JALR_SIGN_MASK - 1) #define to_jalr_t0(offset) \ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0) @@ -96,26 +103,14 @@ do { \ call[1] = to_jalr_t0(offset); \ } while (0) -#define to_jalr_ra(offset) \ - (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_RA) - -#define to_auipc_ra(offset) \ - ((offset & JALR_SIGN_MASK) ? \ - (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_RA) : \ - ((offset & AUIPC_OFFSET_MASK) | AUIPC_RA)) - -#define make_call_ra(caller, callee, call) \ -do { \ - unsigned int offset = \ - (unsigned long) (callee) - (unsigned long) (caller); \ - call[0] = to_auipc_ra(offset); \ - call[1] = to_jalr_ra(offset); \ -} while (0) - /* - * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here. + * Only the jalr insn in the auipc+jalr is patched, so we make it 4 + * bytes here. */ -#define MCOUNT_INSN_SIZE 8 +#define MCOUNT_INSN_SIZE 4 +#define MCOUNT_AUIPC_SIZE 4 +#define MCOUNT_JALR_SIZE 4 +#define MCOUNT_NOP4_SIZE 4 #ifndef __ASSEMBLY__ struct dyn_ftrace; @@ -135,6 +130,9 @@ struct __arch_ftrace_regs { unsigned long sp; unsigned long s0; unsigned long t1; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_tramp; +#endif union { unsigned long args[8]; struct { @@ -146,6 +144,13 @@ struct __arch_ftrace_regs { unsigned long a5; unsigned long a6; unsigned long a7; +#ifdef CONFIG_CC_IS_CLANG + unsigned long t2; + unsigned long t3; + unsigned long t4; + unsigned long t5; + unsigned long t6; +#endif }; }; }; @@ -221,10 +226,13 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct ftrace_regs *fregs); #define ftrace_graph_func ftrace_graph_func +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS static inline void arch_ftrace_set_direct_caller(struct ftrace_regs *fregs, unsigned long addr) { arch_ftrace_regs(fregs)->t1 = addr; } +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ + #endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index e3cbf203cdde..affd63e11b0a 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -105,6 +105,7 @@ #define RISCV_ISA_EXT_ZVFBFWMA 96 #define RISCV_ISA_EXT_ZAAMO 97 #define RISCV_ISA_EXT_ZALRSC 98 +#define RISCV_ISA_EXT_ZICBOP 99 #define RISCV_ISA_EXT_XLINUXENVCFG 127 diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h index 1f690fea0e03..7fe0a379474a 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ #include <uapi/asm/hwprobe.h> -#define RISCV_HWPROBE_MAX_KEY 12 +#define RISCV_HWPROBE_MAX_KEY 13 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { @@ -22,6 +22,7 @@ static inline bool hwprobe_key_is_bitmask(__s64 key) case RISCV_HWPROBE_KEY_IMA_EXT_0: case RISCV_HWPROBE_KEY_CPUPERF_0: case RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0: + case RISCV_HWPROBE_KEY_VENDOR_EXT_SIFIVE_0: return true; } diff --git a/arch/riscv/include/asm/image.h b/arch/riscv/include/asm/image.h index e0b319af3681..8927a6ea1127 100644 --- a/arch/riscv/include/asm/image.h +++ b/arch/riscv/include/asm/image.h @@ -30,6 +30,8 @@ RISCV_HEADER_VERSION_MINOR) #ifndef __ASSEMBLY__ +#define riscv_image_flag_field(flags, field)\ + (((flags) >> field##_SHIFT) & field##_MASK) /** * struct riscv_image_header - riscv kernel image header * @code0: Executable code diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h index 71060a2f838e..d5adbaec1d01 100644 --- a/arch/riscv/include/asm/insn-def.h +++ b/arch/riscv/include/asm/insn-def.h @@ -18,6 +18,13 @@ #define INSN_I_RD_SHIFT 7 #define INSN_I_OPCODE_SHIFT 0 +#define INSN_S_SIMM7_SHIFT 25 +#define INSN_S_RS2_SHIFT 20 +#define INSN_S_RS1_SHIFT 15 +#define INSN_S_FUNC3_SHIFT 12 +#define INSN_S_SIMM5_SHIFT 7 +#define INSN_S_OPCODE_SHIFT 0 + #ifdef __ASSEMBLY__ #ifdef CONFIG_AS_HAS_INSN @@ -30,6 +37,10 @@ .insn i \opcode, \func3, \rd, \rs1, \simm12 .endm + .macro insn_s, opcode, func3, rs2, simm12, rs1 + .insn s \opcode, \func3, \rs2, \simm12(\rs1) + .endm + #else #include <asm/gpr-num.h> @@ -51,10 +62,20 @@ (\simm12 << INSN_I_SIMM12_SHIFT)) .endm + .macro insn_s, opcode, func3, rs2, simm12, rs1 + .4byte ((\opcode << INSN_S_OPCODE_SHIFT) | \ + (\func3 << INSN_S_FUNC3_SHIFT) | \ + (.L__gpr_num_\rs2 << INSN_S_RS2_SHIFT) | \ + (.L__gpr_num_\rs1 << INSN_S_RS1_SHIFT) | \ + ((\simm12 & 0x1f) << INSN_S_SIMM5_SHIFT) | \ + (((\simm12 >> 5) & 0x7f) << INSN_S_SIMM7_SHIFT)) + .endm + #endif #define __INSN_R(...) insn_r __VA_ARGS__ #define __INSN_I(...) insn_i __VA_ARGS__ +#define __INSN_S(...) insn_s __VA_ARGS__ #else /* ! __ASSEMBLY__ */ @@ -66,6 +87,9 @@ #define __INSN_I(opcode, func3, rd, rs1, simm12) \ ".insn i " opcode ", " func3 ", " rd ", " rs1 ", " simm12 "\n" +#define __INSN_S(opcode, func3, rs2, simm12, rs1) \ + ".insn s " opcode ", " func3 ", " rs2 ", " simm12 "(" rs1 ")\n" + #else #include <linux/stringify.h> @@ -92,12 +116,26 @@ " (\\simm12 << " __stringify(INSN_I_SIMM12_SHIFT) "))\n" \ " .endm\n" +#define DEFINE_INSN_S \ + __DEFINE_ASM_GPR_NUMS \ +" .macro insn_s, opcode, func3, rs2, simm12, rs1\n" \ +" .4byte ((\\opcode << " __stringify(INSN_S_OPCODE_SHIFT) ") |" \ +" (\\func3 << " __stringify(INSN_S_FUNC3_SHIFT) ") |" \ +" (.L__gpr_num_\\rs2 << " __stringify(INSN_S_RS2_SHIFT) ") |" \ +" (.L__gpr_num_\\rs1 << " __stringify(INSN_S_RS1_SHIFT) ") |" \ +" ((\\simm12 & 0x1f) << " __stringify(INSN_S_SIMM5_SHIFT) ") |" \ +" (((\\simm12 >> 5) & 0x7f) << " __stringify(INSN_S_SIMM7_SHIFT) "))\n" \ +" .endm\n" + #define UNDEFINE_INSN_R \ " .purgem insn_r\n" #define UNDEFINE_INSN_I \ " .purgem insn_i\n" +#define UNDEFINE_INSN_S \ +" .purgem insn_s\n" + #define __INSN_R(opcode, func3, func7, rd, rs1, rs2) \ DEFINE_INSN_R \ "insn_r " opcode ", " func3 ", " func7 ", " rd ", " rs1 ", " rs2 "\n" \ @@ -108,6 +146,11 @@ "insn_i " opcode ", " func3 ", " rd ", " rs1 ", " simm12 "\n" \ UNDEFINE_INSN_I +#define __INSN_S(opcode, func3, rs2, simm12, rs1) \ + DEFINE_INSN_S \ + "insn_s " opcode ", " func3 ", " rs2 ", " simm12 ", " rs1 "\n" \ + UNDEFINE_INSN_S + #endif #endif /* ! __ASSEMBLY__ */ @@ -120,6 +163,10 @@ __INSN_I(RV_##opcode, RV_##func3, RV_##rd, \ RV_##rs1, RV_##simm12) +#define INSN_S(opcode, func3, rs2, simm12, rs1) \ + __INSN_S(RV_##opcode, RV_##func3, RV_##rs2, \ + RV_##simm12, RV_##rs1) + #define RV_OPCODE(v) __ASM_STR(v) #define RV_FUNC3(v) __ASM_STR(v) #define RV_FUNC7(v) __ASM_STR(v) @@ -133,6 +180,7 @@ #define RV___RS2(v) __RV_REG(v) #define RV_OPCODE_MISC_MEM RV_OPCODE(15) +#define RV_OPCODE_OP_IMM RV_OPCODE(19) #define RV_OPCODE_SYSTEM RV_OPCODE(115) #define HFENCE_VVMA(vaddr, asid) \ @@ -196,6 +244,18 @@ INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \ RS1(base), SIMM12(4)) +#define PREFETCH_I(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(0), \ + SIMM12((offset) & 0xfe0), RS1(base)) + +#define PREFETCH_R(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(1), \ + SIMM12((offset) & 0xfe0), RS1(base)) + +#define PREFETCH_W(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(3), \ + SIMM12((offset) & 0xfe0), RS1(base)) + #define RISCV_PAUSE ".4byte 0x100000f" #define ZAWRS_WRS_NTO ".4byte 0x00d00073" #define ZAWRS_WRS_STO ".4byte 0x01d00073" @@ -203,4 +263,10 @@ #define RISCV_INSN_NOP4 _AC(0x00000013, U) +#ifndef __ASSEMBLY__ +#define nop() __asm__ __volatile__ ("nop") +#define __nops(n) ".rept " #n "\nnop\n.endr\n" +#define nops(n) __asm__ __volatile__ (__nops(n)) +#endif + #endif /* __ASM_INSN_DEF_H */ diff --git a/arch/riscv/include/asm/kexec.h b/arch/riscv/include/asm/kexec.h index 2b56769cb530..b9ee8346cc8c 100644 --- a/arch/riscv/include/asm/kexec.h +++ b/arch/riscv/include/asm/kexec.h @@ -56,6 +56,7 @@ extern riscv_kexec_method riscv_kexec_norelocate; #ifdef CONFIG_KEXEC_FILE extern const struct kexec_file_ops elf_kexec_ops; +extern const struct kexec_file_ops image_kexec_ops; struct purgatory_info; int arch_kexec_apply_relocations_add(struct purgatory_info *pi, @@ -67,6 +68,11 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi, struct kimage; int arch_kimage_file_post_load_cleanup(struct kimage *image); #define arch_kimage_file_post_load_cleanup arch_kimage_file_post_load_cleanup + +int load_extra_segments(struct kimage *image, unsigned long kernel_start, + unsigned long kernel_len, char *initrd, + unsigned long initrd_len, char *cmdline, + unsigned long cmdline_len); #endif #endif diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 188fadc1c21f..7de05db7d3bd 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -184,7 +184,7 @@ static inline int pud_none(pud_t pud) static inline int pud_bad(pud_t pud) { - return !pud_present(pud); + return !pud_present(pud) || (pud_val(pud) & _PAGE_LEAF); } #define pud_leaf pud_leaf @@ -399,6 +399,7 @@ p4d_t *p4d_offset(pgd_t *pgd, unsigned long address); #ifdef CONFIG_TRANSPARENT_HUGEPAGE static inline int pte_devmap(pte_t pte); static inline pte_t pmd_pte(pmd_t pmd); +static inline pte_t pud_pte(pud_t pud); static inline int pmd_devmap(pmd_t pmd) { @@ -407,7 +408,7 @@ static inline int pmd_devmap(pmd_t pmd) static inline int pud_devmap(pud_t pud) { - return 0; + return pte_devmap(pud_pte(pud)); } static inline int pgd_devmap(pgd_t pgd) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index f19240fd018e..438ce7df24c3 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -900,6 +900,103 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, #define pmdp_collapse_flush pmdp_collapse_flush extern pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); + +static inline pud_t pud_wrprotect(pud_t pud) +{ + return pte_pud(pte_wrprotect(pud_pte(pud))); +} + +static inline int pud_trans_huge(pud_t pud) +{ + return pud_leaf(pud); +} + +static inline int pud_dirty(pud_t pud) +{ + return pte_dirty(pud_pte(pud)); +} + +static inline pud_t pud_mkyoung(pud_t pud) +{ + return pte_pud(pte_mkyoung(pud_pte(pud))); +} + +static inline pud_t pud_mkold(pud_t pud) +{ + return pte_pud(pte_mkold(pud_pte(pud))); +} + +static inline pud_t pud_mkdirty(pud_t pud) +{ + return pte_pud(pte_mkdirty(pud_pte(pud))); +} + +static inline pud_t pud_mkclean(pud_t pud) +{ + return pte_pud(pte_mkclean(pud_pte(pud))); +} + +static inline pud_t pud_mkwrite(pud_t pud) +{ + return pte_pud(pte_mkwrite_novma(pud_pte(pud))); +} + +static inline pud_t pud_mkhuge(pud_t pud) +{ + return pud; +} + +static inline pud_t pud_mkdevmap(pud_t pud) +{ + return pte_pud(pte_mkdevmap(pud_pte(pud))); +} + +static inline int pudp_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, + pud_t entry, int dirty) +{ + return ptep_set_access_flags(vma, address, (pte_t *)pudp, pud_pte(entry), dirty); +} + +static inline int pudp_test_and_clear_young(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp) +{ + return ptep_test_and_clear_young(vma, address, (pte_t *)pudp); +} + +static inline int pud_young(pud_t pud) +{ + return pte_young(pud_pte(pud)); +} + +static inline void update_mmu_cache_pud(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp) +{ + pte_t *ptep = (pte_t *)pudp; + + update_mmu_cache(vma, address, ptep); +} + +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + return __pud(atomic_long_xchg((atomic_long_t *)pudp, pud_val(pud))); +} + +static inline pud_t pud_mkinvalid(pud_t pud) +{ + return __pud(pud_val(pud) & ~(_PAGE_PRESENT | _PAGE_PROT_NONE)); +} + +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp); + +static inline pud_t pud_modify(pud_t pud, pgprot_t newprot) +{ + return pte_pud(pte_modify(pud_pte(pud), newprot)); +} + #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* @@ -931,7 +1028,7 @@ extern pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 5f56eb9d114a..24d3af4d3807 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -13,6 +13,9 @@ #include <vdso/processor.h> #include <asm/ptrace.h> +#include <asm/insn-def.h> +#include <asm/alternative-macros.h> +#include <asm/hwcap.h> #define arch_get_mmap_end(addr, len, flags) \ ({ \ @@ -52,7 +55,6 @@ #endif #ifndef __ASSEMBLY__ -#include <linux/cpumask.h> struct task_struct; struct pt_regs; @@ -79,6 +81,10 @@ struct pt_regs; * Thus, the task does not own preempt_v. Any use of Vector will have to * save preempt_v, if dirty, and fallback to non-preemptible kernel-mode * Vector. + * - bit 29: The thread voluntarily calls schedule() while holding an active + * preempt_v. All preempt_v context should be dropped in such case because + * V-regs are caller-saved. Only sstatus.VS=ON is persisted across a + * schedule() call. * - bit 30: The in-kernel preempt_v context is saved, and requries to be * restored when returning to the context that owns the preempt_v. * - bit 31: The in-kernel preempt_v context is dirty, as signaled by the @@ -93,6 +99,7 @@ struct pt_regs; #define RISCV_PREEMPT_V 0x00000100 #define RISCV_PREEMPT_V_DIRTY 0x80000000 #define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000 +#define RISCV_PREEMPT_V_IN_SCHEDULE 0x20000000 /* CPU-specific state of a task */ struct thread_struct { @@ -103,6 +110,7 @@ struct thread_struct { struct __riscv_d_ext_state fstate; unsigned long bad_cause; unsigned long envcfg; + unsigned long sum; u32 riscv_v_flags; u32 vstate_ctrl; struct __riscv_v_ext_state vstate; @@ -136,6 +144,27 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset, #define KSTK_EIP(tsk) (task_pt_regs(tsk)->epc) #define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp) +#define PREFETCH_ASM(x) \ + ALTERNATIVE(__nops(1), PREFETCH_R(x, 0), 0, \ + RISCV_ISA_EXT_ZICBOP, CONFIG_RISCV_ISA_ZICBOP) + +#define PREFETCHW_ASM(x) \ + ALTERNATIVE(__nops(1), PREFETCH_W(x, 0), 0, \ + RISCV_ISA_EXT_ZICBOP, CONFIG_RISCV_ISA_ZICBOP) + +#ifdef CONFIG_RISCV_ISA_ZICBOP +#define ARCH_HAS_PREFETCH +static inline void prefetch(const void *x) +{ + __asm__ __volatile__(PREFETCH_ASM(%0) : : "r" (x) : "memory"); +} + +#define ARCH_HAS_PREFETCHW +static inline void prefetchw(const void *x) +{ + __asm__ __volatile__(PREFETCHW_ASM(%0) : : "r" (x) : "memory"); +} +#endif /* CONFIG_RISCV_ISA_ZICBOP */ /* Do necessary setup to start up a newly executed thread. */ extern void start_thread(struct pt_regs *regs, diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrace.h index 2910231977cb..a7dc0e330757 100644 --- a/arch/riscv/include/asm/ptrace.h +++ b/arch/riscv/include/asm/ptrace.h @@ -175,7 +175,7 @@ static inline unsigned long regs_get_kernel_argument(struct pt_regs *regs, return 0; } -static inline int regs_irqs_disabled(struct pt_regs *regs) +static __always_inline bool regs_irqs_disabled(struct pt_regs *regs) { return !(regs->status & SR_PIE); } diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 3d250824178b..341e74238aa0 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -35,6 +35,7 @@ enum sbi_ext_id { SBI_EXT_DBCN = 0x4442434E, SBI_EXT_STA = 0x535441, SBI_EXT_NACL = 0x4E41434C, + SBI_EXT_FWFT = 0x46574654, /* Experimentals extensions must lie within this range */ SBI_EXT_EXPERIMENTAL_START = 0x08000000, @@ -402,6 +403,33 @@ enum sbi_ext_nacl_feature { #define SBI_NACL_SHMEM_SRET_X(__i) ((__riscv_xlen / 8) * (__i)) #define SBI_NACL_SHMEM_SRET_X_LAST 31 +/* SBI function IDs for FW feature extension */ +#define SBI_EXT_FWFT_SET 0x0 +#define SBI_EXT_FWFT_GET 0x1 + +enum sbi_fwft_feature_t { + SBI_FWFT_MISALIGNED_EXC_DELEG = 0x0, + SBI_FWFT_LANDING_PAD = 0x1, + SBI_FWFT_SHADOW_STACK = 0x2, + SBI_FWFT_DOUBLE_TRAP = 0x3, + SBI_FWFT_PTE_AD_HW_UPDATING = 0x4, + SBI_FWFT_POINTER_MASKING_PMLEN = 0x5, + SBI_FWFT_LOCAL_RESERVED_START = 0x6, + SBI_FWFT_LOCAL_RESERVED_END = 0x3fffffff, + SBI_FWFT_LOCAL_PLATFORM_START = 0x40000000, + SBI_FWFT_LOCAL_PLATFORM_END = 0x7fffffff, + + SBI_FWFT_GLOBAL_RESERVED_START = 0x80000000, + SBI_FWFT_GLOBAL_RESERVED_END = 0xbfffffff, + SBI_FWFT_GLOBAL_PLATFORM_START = 0xc0000000, + SBI_FWFT_GLOBAL_PLATFORM_END = 0xffffffff, +}; + +#define SBI_FWFT_PLATFORM_FEATURE_BIT BIT(30) +#define SBI_FWFT_GLOBAL_FEATURE_BIT BIT(31) + +#define SBI_FWFT_SET_FLAG_LOCK BIT(0) + /* SBI spec version fields */ #define SBI_SPEC_VERSION_DEFAULT 0x1 #define SBI_SPEC_VERSION_MAJOR_SHIFT 24 @@ -419,6 +447,11 @@ enum sbi_ext_nacl_feature { #define SBI_ERR_ALREADY_STARTED -7 #define SBI_ERR_ALREADY_STOPPED -8 #define SBI_ERR_NO_SHMEM -9 +#define SBI_ERR_INVALID_STATE -10 +#define SBI_ERR_BAD_RANGE -11 +#define SBI_ERR_TIMEOUT -12 +#define SBI_ERR_IO -13 +#define SBI_ERR_DENIED_LOCKED -14 extern unsigned long sbi_spec_version; struct sbiret { @@ -470,6 +503,23 @@ int sbi_remote_hfence_vvma_asid(const struct cpumask *cpu_mask, unsigned long asid); long sbi_probe_extension(int ext); +int sbi_fwft_set(u32 feature, unsigned long value, unsigned long flags); +int sbi_fwft_set_cpumask(const cpumask_t *mask, u32 feature, + unsigned long value, unsigned long flags); +/** + * sbi_fwft_set_online_cpus() - Set a feature on all online cpus + * @feature: The feature to be set + * @value: The feature value to be set + * @flags: FWFT feature set flags + * + * Return: 0 on success, appropriate linux error code otherwise. + */ +static inline int sbi_fwft_set_online_cpus(u32 feature, unsigned long value, + unsigned long flags) +{ + return sbi_fwft_set_cpumask(cpu_online_mask, feature, value, flags); +} + /* Check if current SBI specification version is 0.1 or not */ static inline int sbi_spec_is_0_1(void) { @@ -503,11 +553,21 @@ static inline int sbi_err_map_linux_errno(int err) case SBI_SUCCESS: return 0; case SBI_ERR_DENIED: + case SBI_ERR_DENIED_LOCKED: return -EPERM; case SBI_ERR_INVALID_PARAM: + case SBI_ERR_INVALID_STATE: return -EINVAL; + case SBI_ERR_BAD_RANGE: + return -ERANGE; case SBI_ERR_INVALID_ADDRESS: return -EFAULT; + case SBI_ERR_NO_SHMEM: + return -ENOMEM; + case SBI_ERR_TIMEOUT: + return -ETIMEDOUT; + case SBI_ERR_IO: + return -EIO; case SBI_ERR_NOT_SUPPORTED: case SBI_ERR_FAILURE: default: diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index ce0dd0fed764..1a20dd746a49 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -56,6 +56,8 @@ void local_flush_tlb_kernel_range(unsigned long start, unsigned long end); #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); +void flush_pud_tlb_range(struct vm_area_struct *vma, unsigned long start, + unsigned long end); #endif bool arch_tlbbatch_should_defer(struct mm_struct *mm); diff --git a/arch/riscv/include/asm/uaccess.h b/arch/riscv/include/asm/uaccess.h index fee56b0c8058..d472da4450e6 100644 --- a/arch/riscv/include/asm/uaccess.h +++ b/arch/riscv/include/asm/uaccess.h @@ -62,6 +62,19 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigne __asm__ __volatile__ ("csrc sstatus, %0" : : "r" (SR_SUM) : "memory") /* + * This is the smallest unsigned integer type that can fit a value + * (up to 'long long') + */ +#define __inttype(x) __typeof__( \ + __typefits(x, char, \ + __typefits(x, short, \ + __typefits(x, int, \ + __typefits(x, long, 0ULL))))) + +#define __typefits(x, type, not) \ + __builtin_choose_expr(sizeof(x) <= sizeof(type), (unsigned type)0, not) + +/* * The exception table consists of pairs of addresses: the first is the * address of an instruction that is allowed to fault, and the second is * the address at which the program should continue. No registers are @@ -83,27 +96,58 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigne * call. */ -#define __get_user_asm(insn, x, ptr, err) \ +#ifdef CONFIG_CC_HAS_ASM_GOTO_OUTPUT +#define __get_user_asm(insn, x, ptr, label) \ + asm_goto_output( \ + "1:\n" \ + " " insn " %0, %1\n" \ + _ASM_EXTABLE_UACCESS_ERR(1b, %l2, %0) \ + : "=&r" (x) \ + : "m" (*(ptr)) : : label) +#else /* !CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ +#define __get_user_asm(insn, x, ptr, label) \ do { \ - __typeof__(x) __x; \ + long __gua_err = 0; \ __asm__ __volatile__ ( \ "1:\n" \ " " insn " %1, %2\n" \ "2:\n" \ _ASM_EXTABLE_UACCESS_ERR_ZERO(1b, 2b, %0, %1) \ - : "+r" (err), "=&r" (__x) \ + : "+r" (__gua_err), "=&r" (x) \ : "m" (*(ptr))); \ - (x) = __x; \ + if (__gua_err) \ + goto label; \ } while (0) +#endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ #ifdef CONFIG_64BIT -#define __get_user_8(x, ptr, err) \ - __get_user_asm("ld", x, ptr, err) +#define __get_user_8(x, ptr, label) \ + __get_user_asm("ld", x, ptr, label) #else /* !CONFIG_64BIT */ -#define __get_user_8(x, ptr, err) \ + +#ifdef CONFIG_CC_HAS_ASM_GOTO_OUTPUT +#define __get_user_8(x, ptr, label) \ + u32 __user *__ptr = (u32 __user *)(ptr); \ + u32 __lo, __hi; \ + asm_goto_output( \ + "1:\n" \ + " lw %0, %2\n" \ + "2:\n" \ + " lw %1, %3\n" \ + _ASM_EXTABLE_UACCESS_ERR(1b, %l4, %0) \ + _ASM_EXTABLE_UACCESS_ERR(2b, %l4, %0) \ + : "=&r" (__lo), "=r" (__hi) \ + : "m" (__ptr[__LSW]), "m" (__ptr[__MSW]) \ + : : label); \ + (x) = (__typeof__(x))((__typeof__((x) - (x)))( \ + (((u64)__hi << 32) | __lo))); \ + +#else /* !CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ +#define __get_user_8(x, ptr, label) \ do { \ u32 __user *__ptr = (u32 __user *)(ptr); \ u32 __lo, __hi; \ + long __gu8_err = 0; \ __asm__ __volatile__ ( \ "1:\n" \ " lw %1, %3\n" \ @@ -112,35 +156,62 @@ do { \ "3:\n" \ _ASM_EXTABLE_UACCESS_ERR_ZERO(1b, 3b, %0, %1) \ _ASM_EXTABLE_UACCESS_ERR_ZERO(2b, 3b, %0, %1) \ - : "+r" (err), "=&r" (__lo), "=r" (__hi) \ + : "+r" (__gu8_err), "=&r" (__lo), "=r" (__hi) \ : "m" (__ptr[__LSW]), "m" (__ptr[__MSW])); \ - if (err) \ + if (__gu8_err) { \ __hi = 0; \ - (x) = (__typeof__(x))((__typeof__((x)-(x)))( \ + goto label; \ + } \ + (x) = (__typeof__(x))((__typeof__((x) - (x)))( \ (((u64)__hi << 32) | __lo))); \ } while (0) +#endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ + #endif /* CONFIG_64BIT */ -#define __get_user_nocheck(x, __gu_ptr, __gu_err) \ +unsigned long __must_check __asm_copy_to_user_sum_enabled(void __user *to, + const void *from, unsigned long n); +unsigned long __must_check __asm_copy_from_user_sum_enabled(void *to, + const void __user *from, unsigned long n); + +#define __get_user_nocheck(x, __gu_ptr, label) \ do { \ + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && \ + !IS_ALIGNED((uintptr_t)__gu_ptr, sizeof(*__gu_ptr))) { \ + if (__asm_copy_from_user_sum_enabled(&(x), __gu_ptr, sizeof(*__gu_ptr))) \ + goto label; \ + break; \ + } \ switch (sizeof(*__gu_ptr)) { \ case 1: \ - __get_user_asm("lb", (x), __gu_ptr, __gu_err); \ + __get_user_asm("lb", (x), __gu_ptr, label); \ break; \ case 2: \ - __get_user_asm("lh", (x), __gu_ptr, __gu_err); \ + __get_user_asm("lh", (x), __gu_ptr, label); \ break; \ case 4: \ - __get_user_asm("lw", (x), __gu_ptr, __gu_err); \ + __get_user_asm("lw", (x), __gu_ptr, label); \ break; \ case 8: \ - __get_user_8((x), __gu_ptr, __gu_err); \ + __get_user_8((x), __gu_ptr, label); \ break; \ default: \ BUILD_BUG(); \ } \ } while (0) +#define __get_user_error(x, ptr, err) \ +do { \ + __label__ __gu_failed; \ + \ + __get_user_nocheck(x, ptr, __gu_failed); \ + err = 0; \ + break; \ +__gu_failed: \ + x = 0; \ + err = -EFAULT; \ +} while (0) + /** * __get_user: - Get a simple variable from user space, with less checking. * @x: Variable to store result. @@ -165,13 +236,16 @@ do { \ ({ \ const __typeof__(*(ptr)) __user *__gu_ptr = untagged_addr(ptr); \ long __gu_err = 0; \ + __typeof__(x) __gu_val; \ \ __chk_user_ptr(__gu_ptr); \ \ __enable_user_access(); \ - __get_user_nocheck(x, __gu_ptr, __gu_err); \ + __get_user_error(__gu_val, __gu_ptr, __gu_err); \ __disable_user_access(); \ \ + (x) = __gu_val; \ + \ __gu_err; \ }) @@ -201,61 +275,73 @@ do { \ ((x) = (__force __typeof__(x))0, -EFAULT); \ }) -#define __put_user_asm(insn, x, ptr, err) \ +#define __put_user_asm(insn, x, ptr, label) \ do { \ __typeof__(*(ptr)) __x = x; \ - __asm__ __volatile__ ( \ + asm goto( \ "1:\n" \ - " " insn " %z2, %1\n" \ - "2:\n" \ - _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %0) \ - : "+r" (err), "=m" (*(ptr)) \ - : "rJ" (__x)); \ + " " insn " %z0, %1\n" \ + _ASM_EXTABLE(1b, %l2) \ + : : "rJ" (__x), "m"(*(ptr)) : : label); \ } while (0) #ifdef CONFIG_64BIT -#define __put_user_8(x, ptr, err) \ - __put_user_asm("sd", x, ptr, err) +#define __put_user_8(x, ptr, label) \ + __put_user_asm("sd", x, ptr, label) #else /* !CONFIG_64BIT */ -#define __put_user_8(x, ptr, err) \ +#define __put_user_8(x, ptr, label) \ do { \ u32 __user *__ptr = (u32 __user *)(ptr); \ u64 __x = (__typeof__((x)-(x)))(x); \ - __asm__ __volatile__ ( \ + asm goto( \ "1:\n" \ - " sw %z3, %1\n" \ + " sw %z0, %2\n" \ "2:\n" \ - " sw %z4, %2\n" \ - "3:\n" \ - _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %0) \ - _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %0) \ - : "+r" (err), \ - "=m" (__ptr[__LSW]), \ - "=m" (__ptr[__MSW]) \ - : "rJ" (__x), "rJ" (__x >> 32)); \ + " sw %z1, %3\n" \ + _ASM_EXTABLE(1b, %l4) \ + _ASM_EXTABLE(2b, %l4) \ + : : "rJ" (__x), "rJ" (__x >> 32), \ + "m" (__ptr[__LSW]), \ + "m" (__ptr[__MSW]) : : label); \ } while (0) #endif /* CONFIG_64BIT */ -#define __put_user_nocheck(x, __gu_ptr, __pu_err) \ +#define __put_user_nocheck(x, __gu_ptr, label) \ do { \ + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && \ + !IS_ALIGNED((uintptr_t)__gu_ptr, sizeof(*__gu_ptr))) { \ + __inttype(x) val = (__inttype(x))x; \ + if (__asm_copy_to_user_sum_enabled(__gu_ptr, &(val), sizeof(*__gu_ptr))) \ + goto label; \ + break; \ + } \ switch (sizeof(*__gu_ptr)) { \ case 1: \ - __put_user_asm("sb", (x), __gu_ptr, __pu_err); \ + __put_user_asm("sb", (x), __gu_ptr, label); \ break; \ case 2: \ - __put_user_asm("sh", (x), __gu_ptr, __pu_err); \ + __put_user_asm("sh", (x), __gu_ptr, label); \ break; \ case 4: \ - __put_user_asm("sw", (x), __gu_ptr, __pu_err); \ + __put_user_asm("sw", (x), __gu_ptr, label); \ break; \ case 8: \ - __put_user_8((x), __gu_ptr, __pu_err); \ + __put_user_8((x), __gu_ptr, label); \ break; \ default: \ BUILD_BUG(); \ } \ } while (0) +#define __put_user_error(x, ptr, err) \ +do { \ + __label__ err_label; \ + __put_user_nocheck(x, ptr, err_label); \ + break; \ +err_label: \ + (err) = -EFAULT; \ +} while (0) + /** * __put_user: - Write a simple value into user space, with less checking. * @x: Value to copy to user space. @@ -286,7 +372,7 @@ do { \ __chk_user_ptr(__gu_ptr); \ \ __enable_user_access(); \ - __put_user_nocheck(__val, __gu_ptr, __pu_err); \ + __put_user_error(__val, __gu_ptr, __pu_err); \ __disable_user_access(); \ \ __pu_err; \ @@ -351,23 +437,45 @@ unsigned long __must_check clear_user(void __user *to, unsigned long n) } #define __get_kernel_nofault(dst, src, type, err_label) \ -do { \ - long __kr_err = 0; \ - \ - __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \ - if (unlikely(__kr_err)) \ - goto err_label; \ -} while (0) + __get_user_nocheck(*((type *)(dst)), (type *)(src), err_label) #define __put_kernel_nofault(dst, src, type, err_label) \ -do { \ - long __kr_err = 0; \ - \ - __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \ - if (unlikely(__kr_err)) \ - goto err_label; \ + __put_user_nocheck(*((type *)(src)), (type *)(dst), err_label) + +static __must_check __always_inline bool user_access_begin(const void __user *ptr, size_t len) +{ + if (unlikely(!access_ok(ptr, len))) + return 0; + __enable_user_access(); + return 1; +} +#define user_access_begin user_access_begin +#define user_access_end __disable_user_access + +static inline unsigned long user_access_save(void) { return 0UL; } +static inline void user_access_restore(unsigned long enabled) { } + +/* + * We want the unsafe accessors to always be inlined and use + * the error labels - thus the macro games. + */ +#define unsafe_put_user(x, ptr, label) \ + __put_user_nocheck(x, (ptr), label) + +#define unsafe_get_user(x, ptr, label) do { \ + __inttype(*(ptr)) __gu_val; \ + __get_user_nocheck(__gu_val, (ptr), label); \ + (x) = (__force __typeof__(*(ptr)))__gu_val; \ } while (0) +#define unsafe_copy_to_user(_dst, _src, _len, label) \ + if (__asm_copy_to_user_sum_enabled(_dst, _src, _len)) \ + goto label; + +#define unsafe_copy_from_user(_dst, _src, _len, label) \ + if (__asm_copy_from_user_sum_enabled(_dst, _src, _len)) \ + goto label; + #else /* CONFIG_MMU */ #include <asm-generic/uaccess.h> #endif /* CONFIG_MMU */ diff --git a/arch/riscv/include/asm/vdso/getrandom.h b/arch/riscv/include/asm/vdso/getrandom.h new file mode 100644 index 000000000000..8dc92441702a --- /dev/null +++ b/arch/riscv/include/asm/vdso/getrandom.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2025 Xi Ruoyao <xry111@xry111.site>. All Rights Reserved. + */ +#ifndef __ASM_VDSO_GETRANDOM_H +#define __ASM_VDSO_GETRANDOM_H + +#ifndef __ASSEMBLY__ + +#include <asm/unistd.h> + +static __always_inline ssize_t getrandom_syscall(void *_buffer, size_t _len, unsigned int _flags) +{ + register long ret asm("a0"); + register long nr asm("a7") = __NR_getrandom; + register void *buffer asm("a0") = _buffer; + register size_t len asm("a1") = _len; + register unsigned int flags asm("a2") = _flags; + + asm volatile ("ecall\n" + : "+r" (ret) + : "r" (nr), "r" (buffer), "r" (len), "r" (flags) + : "memory"); + + return ret; +} + +#endif /* !__ASSEMBLY__ */ + +#endif /* __ASM_VDSO_GETRANDOM_H */ diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index e8a83f55be2b..45c9b426fcc5 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -120,6 +120,11 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline bool riscv_v_is_on(void) +{ + return !!(csr_read(CSR_SSTATUS) & SR_VS); +} + static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) { asm volatile ( @@ -366,6 +371,11 @@ static inline void __switch_to_vector(struct task_struct *prev, struct pt_regs *regs; if (riscv_preempt_v_started(prev)) { + if (riscv_v_is_on()) { + WARN_ON(prev->thread.riscv_v_flags & RISCV_V_CTX_DEPTH_MASK); + riscv_v_disable(); + prev->thread.riscv_v_flags |= RISCV_PREEMPT_V_IN_SCHEDULE; + } if (riscv_preempt_v_dirty(prev)) { __riscv_v_vstate_save(&prev->thread.kernel_vstate, prev->thread.kernel_vstate.datap); @@ -376,10 +386,16 @@ static inline void __switch_to_vector(struct task_struct *prev, riscv_v_vstate_save(&prev->thread.vstate, regs); } - if (riscv_preempt_v_started(next)) - riscv_preempt_v_set_restore(next); - else + if (riscv_preempt_v_started(next)) { + if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) { + next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE; + riscv_v_enable(); + } else { + riscv_preempt_v_set_restore(next); + } + } else { riscv_v_vstate_set_restore(next, task_pt_regs(next)); + } } void riscv_v_vstate_ctrl_init(struct task_struct *tsk); diff --git a/arch/riscv/include/asm/vendor_extensions/sifive.h b/arch/riscv/include/asm/vendor_extensions/sifive.h new file mode 100644 index 000000000000..ac00e500361c --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/sifive.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_SIFIVE_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_SIFIVE_H + +#include <asm/vendor_extensions.h> + +#include <linux/types.h> + +#define RISCV_ISA_VENDOR_EXT_XSFVQMACCDOD 0 +#define RISCV_ISA_VENDOR_EXT_XSFVQMACCQOQ 1 +#define RISCV_ISA_VENDOR_EXT_XSFVFNRCLIPXFQF 2 +#define RISCV_ISA_VENDOR_EXT_XSFVFWMACCQQQ 3 + +extern struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_sifive; + +#endif diff --git a/arch/riscv/include/asm/vendor_extensions/sifive_hwprobe.h b/arch/riscv/include/asm/vendor_extensions/sifive_hwprobe.h new file mode 100644 index 000000000000..90a61abd033c --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/sifive_hwprobe.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_SIFIVE_HWPROBE_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_SIFIVE_HWPROBE_H + +#include <linux/cpumask.h> + +#include <uapi/asm/hwprobe.h> + +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_SIFIVE +void hwprobe_isa_vendor_ext_sifive_0(struct riscv_hwprobe *pair, const struct cpumask *cpus); +#else +static inline void hwprobe_isa_vendor_ext_sifive_0(struct riscv_hwprobe *pair, + const struct cpumask *cpus) +{ + pair->value = 0; +} +#endif + +#endif diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 3c2fce939673..aaf6ad970499 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -81,6 +81,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_EXT_ZICBOM (1ULL << 55) #define RISCV_HWPROBE_EXT_ZAAMO (1ULL << 56) #define RISCV_HWPROBE_EXT_ZALRSC (1ULL << 57) +#define RISCV_HWPROBE_EXT_ZABHA (1ULL << 58) #define RISCV_HWPROBE_KEY_CPUPERF_0 5 #define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) #define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) @@ -104,6 +105,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_VECTOR_UNSUPPORTED 4 #define RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 11 #define RISCV_HWPROBE_KEY_ZICBOM_BLOCK_SIZE 12 +#define RISCV_HWPROBE_KEY_VENDOR_EXT_SIFIVE_0 13 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ /* Flags */ diff --git a/arch/riscv/include/uapi/asm/vendor/sifive.h b/arch/riscv/include/uapi/asm/vendor/sifive.h new file mode 100644 index 000000000000..9f3278a4b298 --- /dev/null +++ b/arch/riscv/include/uapi/asm/vendor/sifive.h @@ -0,0 +1,6 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ + +#define RISCV_HWPROBE_VENDOR_EXT_XSFVQMACCDOD (1 << 0) +#define RISCV_HWPROBE_VENDOR_EXT_XSFVQMACCQOQ (1 << 1) +#define RISCV_HWPROBE_VENDOR_EXT_XSFVFNRCLIPXFQF (1 << 2) +#define RISCV_HWPROBE_VENDOR_EXT_XSFVFWMACCQQQ (1 << 3) diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index f7480c9c6f8d..c7b542573407 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -43,7 +43,7 @@ CFLAGS_sbi_ecall.o += -D__NO_FORTIFY endif endif -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds obj-y += head.o obj-y += soc.o @@ -107,7 +107,7 @@ obj-$(CONFIG_HOTPLUG_CPU) += cpu-hotplug.o obj-$(CONFIG_PARAVIRT) += paravirt.o obj-$(CONFIG_KGDB) += kgdb.o obj-$(CONFIG_KEXEC_CORE) += kexec_relocate.o crash_save_regs.o machine_kexec.o -obj-$(CONFIG_KEXEC_FILE) += elf_kexec.o machine_kexec_file.o +obj-$(CONFIG_KEXEC_FILE) += kexec_elf.o kexec_image.o machine_kexec_file.o obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_VMCORE_INFO) += vmcore_info.o diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index 16490755304e..6e8c0d6feae9 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -34,6 +34,7 @@ void asm_offsets(void) OFFSET(TASK_THREAD_S9, task_struct, thread.s[9]); OFFSET(TASK_THREAD_S10, task_struct, thread.s[10]); OFFSET(TASK_THREAD_S11, task_struct, thread.s[11]); + OFFSET(TASK_THREAD_SUM, task_struct, thread.sum); OFFSET(TASK_TI_CPU, task_struct, thread_info.cpu); OFFSET(TASK_TI_PREEMPT_COUNT, task_struct, thread_info.preempt_count); @@ -346,6 +347,10 @@ void asm_offsets(void) offsetof(struct task_struct, thread.s[11]) - offsetof(struct task_struct, thread.ra) ); + DEFINE(TASK_THREAD_SUM_RA, + offsetof(struct task_struct, thread.sum) + - offsetof(struct task_struct, thread.ra) + ); DEFINE(TASK_THREAD_F0_F0, offsetof(struct task_struct, thread.fstate.f[0]) @@ -493,6 +498,12 @@ void asm_offsets(void) DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe), STACK_ALIGN)); OFFSET(STACKFRAME_FP, stackframe, fp); OFFSET(STACKFRAME_RA, stackframe, ra); +#ifdef CONFIG_FUNCTION_TRACER + DEFINE(FTRACE_OPS_FUNC, offsetof(struct ftrace_ops, func)); +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + DEFINE(FTRACE_OPS_DIRECT_CALL, offsetof(struct ftrace_ops, direct_call)); +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ +#endif #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS DEFINE(FREGS_SIZE_ON_STACK, ALIGN(sizeof(struct __arch_ftrace_regs), STACK_ALIGN)); @@ -501,6 +512,13 @@ void asm_offsets(void) DEFINE(FREGS_SP, offsetof(struct __arch_ftrace_regs, sp)); DEFINE(FREGS_S0, offsetof(struct __arch_ftrace_regs, s0)); DEFINE(FREGS_T1, offsetof(struct __arch_ftrace_regs, t1)); +#ifdef CONFIG_CC_IS_CLANG + DEFINE(FREGS_T2, offsetof(struct __arch_ftrace_regs, t2)); + DEFINE(FREGS_T3, offsetof(struct __arch_ftrace_regs, t3)); + DEFINE(FREGS_T4, offsetof(struct __arch_ftrace_regs, t4)); + DEFINE(FREGS_T5, offsetof(struct __arch_ftrace_regs, t5)); + DEFINE(FREGS_T6, offsetof(struct __arch_ftrace_regs, t6)); +#endif DEFINE(FREGS_A0, offsetof(struct __arch_ftrace_regs, a0)); DEFINE(FREGS_A1, offsetof(struct __arch_ftrace_regs, a1)); DEFINE(FREGS_A2, offsetof(struct __arch_ftrace_regs, a2)); diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 2054f6c4b0ae..743d53415572 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -32,6 +32,7 @@ #define NUM_ALPHA_EXTS ('z' - 'a' + 1) static bool any_cpu_has_zicboz; +static bool any_cpu_has_zicbop; static bool any_cpu_has_zicbom; unsigned long elf_hwcap __read_mostly; @@ -119,6 +120,21 @@ static int riscv_ext_zicboz_validate(const struct riscv_isa_ext_data *data, return 0; } +static int riscv_ext_zicbop_validate(const struct riscv_isa_ext_data *data, + const unsigned long *isa_bitmap) +{ + if (!riscv_cbop_block_size) { + pr_err("Zicbop detected in ISA string, disabling as no cbop-block-size found\n"); + return -EINVAL; + } + if (!is_power_of_2(riscv_cbop_block_size)) { + pr_err("Zicbop disabled as cbop-block-size present, but is not a power-of-2\n"); + return -EINVAL; + } + any_cpu_has_zicbop = true; + return 0; +} + static int riscv_ext_f_validate(const struct riscv_isa_ext_data *data, const unsigned long *isa_bitmap) { @@ -442,6 +458,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_SUPERSET_VALIDATE(v, RISCV_ISA_EXT_v, riscv_v_exts, riscv_ext_vector_float_validate), __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h), __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicbom, RISCV_ISA_EXT_ZICBOM, riscv_xlinuxenvcfg_exts, riscv_ext_zicbom_validate), + __RISCV_ISA_EXT_DATA_VALIDATE(zicbop, RISCV_ISA_EXT_ZICBOP, riscv_ext_zicbop_validate), __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicboz, RISCV_ISA_EXT_ZICBOZ, riscv_xlinuxenvcfg_exts, riscv_ext_zicboz_validate), __RISCV_ISA_EXT_DATA(ziccrse, RISCV_ISA_EXT_ZICCRSE), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), @@ -1112,6 +1129,10 @@ void __init riscv_user_isa_enable(void) current->thread.envcfg |= ENVCFG_CBCFE; else if (any_cpu_has_zicbom) pr_warn("Zicbom disabled as it is unavailable on some harts\n"); + + if (!riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICBOP) && + any_cpu_has_zicbop) + pr_warn("Zicbop disabled as it is unavailable on some harts\n"); } #ifdef CONFIG_RISCV_ALTERNATIVE diff --git a/arch/riscv/kernel/elf_kexec.c b/arch/riscv/kernel/elf_kexec.c deleted file mode 100644 index e783a72d051f..000000000000 --- a/arch/riscv/kernel/elf_kexec.c +++ /dev/null @@ -1,485 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Load ELF vmlinux file for the kexec_file_load syscall. - * - * Copyright (C) 2021 Huawei Technologies Co, Ltd. - * - * Author: Liao Chang (liaochang1@huawei.com) - * - * Based on kexec-tools' kexec-elf-riscv.c, heavily modified - * for kernel. - */ - -#define pr_fmt(fmt) "kexec_image: " fmt - -#include <linux/elf.h> -#include <linux/kexec.h> -#include <linux/slab.h> -#include <linux/of.h> -#include <linux/libfdt.h> -#include <linux/types.h> -#include <linux/memblock.h> -#include <linux/vmalloc.h> -#include <asm/setup.h> - -int arch_kimage_file_post_load_cleanup(struct kimage *image) -{ - kvfree(image->arch.fdt); - image->arch.fdt = NULL; - - vfree(image->elf_headers); - image->elf_headers = NULL; - image->elf_headers_sz = 0; - - return kexec_image_post_load_cleanup_default(image); -} - -static int riscv_kexec_elf_load(struct kimage *image, struct elfhdr *ehdr, - struct kexec_elf_info *elf_info, unsigned long old_pbase, - unsigned long new_pbase) -{ - int i; - int ret = 0; - size_t size; - struct kexec_buf kbuf; - const struct elf_phdr *phdr; - - kbuf.image = image; - - for (i = 0; i < ehdr->e_phnum; i++) { - phdr = &elf_info->proghdrs[i]; - if (phdr->p_type != PT_LOAD) - continue; - - size = phdr->p_filesz; - if (size > phdr->p_memsz) - size = phdr->p_memsz; - - kbuf.buffer = (void *) elf_info->buffer + phdr->p_offset; - kbuf.bufsz = size; - kbuf.buf_align = phdr->p_align; - kbuf.mem = phdr->p_paddr - old_pbase + new_pbase; - kbuf.memsz = phdr->p_memsz; - kbuf.top_down = false; - ret = kexec_add_buffer(&kbuf); - if (ret) - break; - } - - return ret; -} - -/* - * Go through the available phsyical memory regions and find one that hold - * an image of the specified size. - */ -static int elf_find_pbase(struct kimage *image, unsigned long kernel_len, - struct elfhdr *ehdr, struct kexec_elf_info *elf_info, - unsigned long *old_pbase, unsigned long *new_pbase) -{ - int i; - int ret; - struct kexec_buf kbuf; - const struct elf_phdr *phdr; - unsigned long lowest_paddr = ULONG_MAX; - unsigned long lowest_vaddr = ULONG_MAX; - - for (i = 0; i < ehdr->e_phnum; i++) { - phdr = &elf_info->proghdrs[i]; - if (phdr->p_type != PT_LOAD) - continue; - - if (lowest_paddr > phdr->p_paddr) - lowest_paddr = phdr->p_paddr; - - if (lowest_vaddr > phdr->p_vaddr) - lowest_vaddr = phdr->p_vaddr; - } - - kbuf.image = image; - kbuf.buf_min = lowest_paddr; - kbuf.buf_max = ULONG_MAX; - - /* - * Current riscv boot protocol requires 2MB alignment for - * RV64 and 4MB alignment for RV32 - * - */ - kbuf.buf_align = PMD_SIZE; - kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; - kbuf.memsz = ALIGN(kernel_len, PAGE_SIZE); - kbuf.top_down = false; - ret = arch_kexec_locate_mem_hole(&kbuf); - if (!ret) { - *old_pbase = lowest_paddr; - *new_pbase = kbuf.mem; - image->start = ehdr->e_entry - lowest_vaddr + kbuf.mem; - } - return ret; -} - -#ifdef CONFIG_CRASH_DUMP -static int get_nr_ram_ranges_callback(struct resource *res, void *arg) -{ - unsigned int *nr_ranges = arg; - - (*nr_ranges)++; - return 0; -} - -static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) -{ - struct crash_mem *cmem = arg; - - cmem->ranges[cmem->nr_ranges].start = res->start; - cmem->ranges[cmem->nr_ranges].end = res->end; - cmem->nr_ranges++; - - return 0; -} - -static int prepare_elf_headers(void **addr, unsigned long *sz) -{ - struct crash_mem *cmem; - unsigned int nr_ranges; - int ret; - - nr_ranges = 1; /* For exclusion of crashkernel region */ - walk_system_ram_res(0, -1, &nr_ranges, get_nr_ram_ranges_callback); - - cmem = kmalloc(struct_size(cmem, ranges, nr_ranges), GFP_KERNEL); - if (!cmem) - return -ENOMEM; - - cmem->max_nr_ranges = nr_ranges; - cmem->nr_ranges = 0; - ret = walk_system_ram_res(0, -1, cmem, prepare_elf64_ram_headers_callback); - if (ret) - goto out; - - /* Exclude crashkernel region */ - ret = crash_exclude_mem_range(cmem, crashk_res.start, crashk_res.end); - if (!ret) - ret = crash_prepare_elf64_headers(cmem, true, addr, sz); - -out: - kfree(cmem); - return ret; -} - -static char *setup_kdump_cmdline(struct kimage *image, char *cmdline, - unsigned long cmdline_len) -{ - int elfcorehdr_strlen; - char *cmdline_ptr; - - cmdline_ptr = kzalloc(COMMAND_LINE_SIZE, GFP_KERNEL); - if (!cmdline_ptr) - return NULL; - - elfcorehdr_strlen = sprintf(cmdline_ptr, "elfcorehdr=0x%lx ", - image->elf_load_addr); - - if (elfcorehdr_strlen + cmdline_len > COMMAND_LINE_SIZE) { - pr_err("Appending elfcorehdr=<addr> exceeds cmdline size\n"); - kfree(cmdline_ptr); - return NULL; - } - - memcpy(cmdline_ptr + elfcorehdr_strlen, cmdline, cmdline_len); - /* Ensure it's nul terminated */ - cmdline_ptr[COMMAND_LINE_SIZE - 1] = '\0'; - return cmdline_ptr; -} -#endif - -static void *elf_kexec_load(struct kimage *image, char *kernel_buf, - unsigned long kernel_len, char *initrd, - unsigned long initrd_len, char *cmdline, - unsigned long cmdline_len) -{ - int ret; - void *fdt; - unsigned long old_kernel_pbase = ULONG_MAX; - unsigned long new_kernel_pbase = 0UL; - unsigned long initrd_pbase = 0UL; - unsigned long kernel_start; - struct elfhdr ehdr; - struct kexec_buf kbuf; - struct kexec_elf_info elf_info; - char *modified_cmdline = NULL; - - ret = kexec_build_elf_info(kernel_buf, kernel_len, &ehdr, &elf_info); - if (ret) - return ERR_PTR(ret); - - ret = elf_find_pbase(image, kernel_len, &ehdr, &elf_info, - &old_kernel_pbase, &new_kernel_pbase); - if (ret) - goto out; - kernel_start = image->start; - - /* Add the kernel binary to the image */ - ret = riscv_kexec_elf_load(image, &ehdr, &elf_info, - old_kernel_pbase, new_kernel_pbase); - if (ret) - goto out; - - kbuf.image = image; - kbuf.buf_min = new_kernel_pbase + kernel_len; - kbuf.buf_max = ULONG_MAX; - -#ifdef CONFIG_CRASH_DUMP - /* Add elfcorehdr */ - if (image->type == KEXEC_TYPE_CRASH) { - void *headers; - unsigned long headers_sz; - ret = prepare_elf_headers(&headers, &headers_sz); - if (ret) { - pr_err("Preparing elf core header failed\n"); - goto out; - } - - kbuf.buffer = headers; - kbuf.bufsz = headers_sz; - kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; - kbuf.memsz = headers_sz; - kbuf.buf_align = ELF_CORE_HEADER_ALIGN; - kbuf.top_down = true; - - ret = kexec_add_buffer(&kbuf); - if (ret) { - vfree(headers); - goto out; - } - image->elf_headers = headers; - image->elf_load_addr = kbuf.mem; - image->elf_headers_sz = headers_sz; - - kexec_dprintk("Loaded elf core header at 0x%lx bufsz=0x%lx memsz=0x%lx\n", - image->elf_load_addr, kbuf.bufsz, kbuf.memsz); - - /* Setup cmdline for kdump kernel case */ - modified_cmdline = setup_kdump_cmdline(image, cmdline, - cmdline_len); - if (!modified_cmdline) { - pr_err("Setting up cmdline for kdump kernel failed\n"); - ret = -EINVAL; - goto out; - } - cmdline = modified_cmdline; - } -#endif - -#ifdef CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY - /* Add purgatory to the image */ - kbuf.top_down = true; - kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; - ret = kexec_load_purgatory(image, &kbuf); - if (ret) { - pr_err("Error loading purgatory ret=%d\n", ret); - goto out; - } - kexec_dprintk("Loaded purgatory at 0x%lx\n", kbuf.mem); - - ret = kexec_purgatory_get_set_symbol(image, "riscv_kernel_entry", - &kernel_start, - sizeof(kernel_start), 0); - if (ret) - pr_err("Error update purgatory ret=%d\n", ret); -#endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */ - - /* Add the initrd to the image */ - if (initrd != NULL) { - kbuf.buffer = initrd; - kbuf.bufsz = kbuf.memsz = initrd_len; - kbuf.buf_align = PAGE_SIZE; - kbuf.top_down = true; - kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; - ret = kexec_add_buffer(&kbuf); - if (ret) - goto out; - initrd_pbase = kbuf.mem; - kexec_dprintk("Loaded initrd at 0x%lx\n", initrd_pbase); - } - - /* Add the DTB to the image */ - fdt = of_kexec_alloc_and_setup_fdt(image, initrd_pbase, - initrd_len, cmdline, 0); - if (!fdt) { - pr_err("Error setting up the new device tree.\n"); - ret = -EINVAL; - goto out; - } - - fdt_pack(fdt); - kbuf.buffer = fdt; - kbuf.bufsz = kbuf.memsz = fdt_totalsize(fdt); - kbuf.buf_align = PAGE_SIZE; - kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; - kbuf.top_down = true; - ret = kexec_add_buffer(&kbuf); - if (ret) { - pr_err("Error add DTB kbuf ret=%d\n", ret); - goto out_free_fdt; - } - /* Cache the fdt buffer address for memory cleanup */ - image->arch.fdt = fdt; - kexec_dprintk("Loaded device tree at 0x%lx\n", kbuf.mem); - goto out; - -out_free_fdt: - kvfree(fdt); -out: - kfree(modified_cmdline); - kexec_free_elf_info(&elf_info); - return ret ? ERR_PTR(ret) : NULL; -} - -#define RV_X(x, s, n) (((x) >> (s)) & ((1 << (n)) - 1)) -#define RISCV_IMM_BITS 12 -#define RISCV_IMM_REACH (1LL << RISCV_IMM_BITS) -#define RISCV_CONST_HIGH_PART(x) \ - (((x) + (RISCV_IMM_REACH >> 1)) & ~(RISCV_IMM_REACH - 1)) -#define RISCV_CONST_LOW_PART(x) ((x) - RISCV_CONST_HIGH_PART(x)) - -#define ENCODE_ITYPE_IMM(x) \ - (RV_X(x, 0, 12) << 20) -#define ENCODE_BTYPE_IMM(x) \ - ((RV_X(x, 1, 4) << 8) | (RV_X(x, 5, 6) << 25) | \ - (RV_X(x, 11, 1) << 7) | (RV_X(x, 12, 1) << 31)) -#define ENCODE_UTYPE_IMM(x) \ - (RV_X(x, 12, 20) << 12) -#define ENCODE_JTYPE_IMM(x) \ - ((RV_X(x, 1, 10) << 21) | (RV_X(x, 11, 1) << 20) | \ - (RV_X(x, 12, 8) << 12) | (RV_X(x, 20, 1) << 31)) -#define ENCODE_CBTYPE_IMM(x) \ - ((RV_X(x, 1, 2) << 3) | (RV_X(x, 3, 2) << 10) | (RV_X(x, 5, 1) << 2) | \ - (RV_X(x, 6, 2) << 5) | (RV_X(x, 8, 1) << 12)) -#define ENCODE_CJTYPE_IMM(x) \ - ((RV_X(x, 1, 3) << 3) | (RV_X(x, 4, 1) << 11) | (RV_X(x, 5, 1) << 2) | \ - (RV_X(x, 6, 1) << 7) | (RV_X(x, 7, 1) << 6) | (RV_X(x, 8, 2) << 9) | \ - (RV_X(x, 10, 1) << 8) | (RV_X(x, 11, 1) << 12)) -#define ENCODE_UJTYPE_IMM(x) \ - (ENCODE_UTYPE_IMM(RISCV_CONST_HIGH_PART(x)) | \ - (ENCODE_ITYPE_IMM(RISCV_CONST_LOW_PART(x)) << 32)) -#define ENCODE_UITYPE_IMM(x) \ - (ENCODE_UTYPE_IMM(x) | (ENCODE_ITYPE_IMM(x) << 32)) - -#define CLEAN_IMM(type, x) \ - ((~ENCODE_##type##_IMM((uint64_t)(-1))) & (x)) - -int arch_kexec_apply_relocations_add(struct purgatory_info *pi, - Elf_Shdr *section, - const Elf_Shdr *relsec, - const Elf_Shdr *symtab) -{ - const char *strtab, *name, *shstrtab; - const Elf_Shdr *sechdrs; - Elf64_Rela *relas; - int i, r_type; - - /* String & section header string table */ - sechdrs = (void *)pi->ehdr + pi->ehdr->e_shoff; - strtab = (char *)pi->ehdr + sechdrs[symtab->sh_link].sh_offset; - shstrtab = (char *)pi->ehdr + sechdrs[pi->ehdr->e_shstrndx].sh_offset; - - relas = (void *)pi->ehdr + relsec->sh_offset; - - for (i = 0; i < relsec->sh_size / sizeof(*relas); i++) { - const Elf_Sym *sym; /* symbol to relocate */ - unsigned long addr; /* final location after relocation */ - unsigned long val; /* relocated symbol value */ - unsigned long sec_base; /* relocated symbol value */ - void *loc; /* tmp location to modify */ - - sym = (void *)pi->ehdr + symtab->sh_offset; - sym += ELF64_R_SYM(relas[i].r_info); - - if (sym->st_name) - name = strtab + sym->st_name; - else - name = shstrtab + sechdrs[sym->st_shndx].sh_name; - - loc = pi->purgatory_buf; - loc += section->sh_offset; - loc += relas[i].r_offset; - - if (sym->st_shndx == SHN_ABS) - sec_base = 0; - else if (sym->st_shndx >= pi->ehdr->e_shnum) { - pr_err("Invalid section %d for symbol %s\n", - sym->st_shndx, name); - return -ENOEXEC; - } else - sec_base = pi->sechdrs[sym->st_shndx].sh_addr; - - val = sym->st_value; - val += sec_base; - val += relas[i].r_addend; - - addr = section->sh_addr + relas[i].r_offset; - - r_type = ELF64_R_TYPE(relas[i].r_info); - - switch (r_type) { - case R_RISCV_BRANCH: - *(u32 *)loc = CLEAN_IMM(BTYPE, *(u32 *)loc) | - ENCODE_BTYPE_IMM(val - addr); - break; - case R_RISCV_JAL: - *(u32 *)loc = CLEAN_IMM(JTYPE, *(u32 *)loc) | - ENCODE_JTYPE_IMM(val - addr); - break; - /* - * With no R_RISCV_PCREL_LO12_S, R_RISCV_PCREL_LO12_I - * sym is expected to be next to R_RISCV_PCREL_HI20 - * in purgatory relsec. Handle it like R_RISCV_CALL - * sym, instead of searching the whole relsec. - */ - case R_RISCV_PCREL_HI20: - case R_RISCV_CALL_PLT: - case R_RISCV_CALL: - *(u64 *)loc = CLEAN_IMM(UITYPE, *(u64 *)loc) | - ENCODE_UJTYPE_IMM(val - addr); - break; - case R_RISCV_RVC_BRANCH: - *(u32 *)loc = CLEAN_IMM(CBTYPE, *(u32 *)loc) | - ENCODE_CBTYPE_IMM(val - addr); - break; - case R_RISCV_RVC_JUMP: - *(u32 *)loc = CLEAN_IMM(CJTYPE, *(u32 *)loc) | - ENCODE_CJTYPE_IMM(val - addr); - break; - case R_RISCV_ADD16: - *(u16 *)loc += val; - break; - case R_RISCV_SUB16: - *(u16 *)loc -= val; - break; - case R_RISCV_ADD32: - *(u32 *)loc += val; - break; - case R_RISCV_SUB32: - *(u32 *)loc -= val; - break; - /* It has been applied by R_RISCV_PCREL_HI20 sym */ - case R_RISCV_PCREL_LO12_I: - case R_RISCV_ALIGN: - case R_RISCV_RELAX: - break; - case R_RISCV_64: - *(u64 *)loc = val; - break; - default: - pr_err("Unknown rela relocation: %d\n", r_type); - return -ENOEXEC; - } - } - return 0; -} - -const struct kexec_file_ops elf_kexec_ops = { - .probe = kexec_elf_probe, - .load = elf_kexec_load, -}; diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 0fb338000c6d..75656afa2d6b 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -401,9 +401,18 @@ SYM_FUNC_START(__switch_to) REG_S s9, TASK_THREAD_S9_RA(a3) REG_S s10, TASK_THREAD_S10_RA(a3) REG_S s11, TASK_THREAD_S11_RA(a3) + + /* save the user space access flag */ + csrr s0, CSR_STATUS + REG_S s0, TASK_THREAD_SUM_RA(a3) + /* Save the kernel shadow call stack pointer */ scs_save_current /* Restore context from next->thread */ + REG_L s0, TASK_THREAD_SUM_RA(a4) + li s1, SR_SUM + and s0, s0, s1 + csrs CSR_STATUS, s0 REG_L ra, TASK_THREAD_RA_RA(a4) REG_L sp, TASK_THREAD_SP_RA(a4) REG_L s0, TASK_THREAD_S0_RA(a4) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index 674dcdfae7a1..4c6c24380cfd 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -8,98 +8,129 @@ #include <linux/ftrace.h> #include <linux/uaccess.h> #include <linux/memory.h> +#include <linux/irqflags.h> #include <linux/stop_machine.h> #include <asm/cacheflush.h> #include <asm/text-patching.h> #ifdef CONFIG_DYNAMIC_FTRACE -void ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +unsigned long ftrace_call_adjust(unsigned long addr) { - mutex_lock(&text_mutex); + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) + return addr + 8 + MCOUNT_AUIPC_SIZE; - /* - * The code sequences we use for ftrace can't be patched while the - * kernel is running, so we need to use stop_machine() to modify them - * for now. This doesn't play nice with text_mutex, we use this flag - * to elide the check. - */ - riscv_patch_in_stop_machine = true; + return addr + MCOUNT_AUIPC_SIZE; +} + +unsigned long arch_ftrace_get_symaddr(unsigned long fentry_ip) +{ + return fentry_ip - MCOUNT_AUIPC_SIZE; } -void ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) +void arch_ftrace_update_code(int command) { - riscv_patch_in_stop_machine = false; + mutex_lock(&text_mutex); + command |= FTRACE_MAY_SLEEP; + ftrace_modify_all_code(command); mutex_unlock(&text_mutex); + flush_icache_all(); } -static int ftrace_check_current_call(unsigned long hook_pos, - unsigned int *expected) +static int __ftrace_modify_call(unsigned long source, unsigned long target, bool validate) { + unsigned int call[2], offset; unsigned int replaced[2]; - unsigned int nops[2] = {RISCV_INSN_NOP4, RISCV_INSN_NOP4}; - /* we expect nops at the hook position */ - if (!expected) - expected = nops; + offset = target - source; + call[1] = to_jalr_t0(offset); - /* - * Read the text we want to modify; - * return must be -EFAULT on read error - */ - if (copy_from_kernel_nofault(replaced, (void *)hook_pos, - MCOUNT_INSN_SIZE)) - return -EFAULT; - - /* - * Make sure it is what we expect it to be; - * return must be -EINVAL on failed comparison - */ - if (memcmp(expected, replaced, sizeof(replaced))) { - pr_err("%p: expected (%08x %08x) but got (%08x %08x)\n", - (void *)hook_pos, expected[0], expected[1], replaced[0], - replaced[1]); - return -EINVAL; + if (validate) { + call[0] = to_auipc_t0(offset); + /* + * Read the text we want to modify; + * return must be -EFAULT on read error + */ + if (copy_from_kernel_nofault(replaced, (void *)source, 2 * MCOUNT_INSN_SIZE)) + return -EFAULT; + + if (replaced[0] != call[0]) { + pr_err("%p: expected (%08x) but got (%08x)\n", + (void *)source, call[0], replaced[0]); + return -EINVAL; + } } + /* Replace the jalr at once. Return -EPERM on write error. */ + if (patch_insn_write((void *)(source + MCOUNT_AUIPC_SIZE), call + 1, MCOUNT_JALR_SIZE)) + return -EPERM; + return 0; } -static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, - bool enable, bool ra) +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS +static const struct ftrace_ops *riscv64_rec_get_ops(struct dyn_ftrace *rec) { - unsigned int call[2]; - unsigned int nops[2] = {RISCV_INSN_NOP4, RISCV_INSN_NOP4}; + const struct ftrace_ops *ops = NULL; - if (ra) - make_call_ra(hook_pos, target, call); - else - make_call_t0(hook_pos, target, call); + if (rec->flags & FTRACE_FL_CALL_OPS_EN) { + ops = ftrace_find_unique_ops(rec); + WARN_ON_ONCE(!ops); + } - /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ - if (patch_insn_write((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) - return -EPERM; + if (!ops) + ops = &ftrace_list_ops; - return 0; + return ops; +} + +static int ftrace_rec_set_ops(const struct dyn_ftrace *rec, const struct ftrace_ops *ops) +{ + unsigned long literal = ALIGN_DOWN(rec->ip - 12, 8); + + return patch_text_nosync((void *)literal, &ops, sizeof(ops)); +} + +static int ftrace_rec_set_nop_ops(struct dyn_ftrace *rec) +{ + return ftrace_rec_set_ops(rec, &ftrace_nop_ops); +} + +static int ftrace_rec_update_ops(struct dyn_ftrace *rec) +{ + return ftrace_rec_set_ops(rec, riscv64_rec_get_ops(rec)); } +#else +static int ftrace_rec_set_nop_ops(struct dyn_ftrace *rec) { return 0; } +static int ftrace_rec_update_ops(struct dyn_ftrace *rec) { return 0; } +#endif int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) { - unsigned int call[2]; + unsigned long distance, orig_addr, pc = rec->ip - MCOUNT_AUIPC_SIZE; + int ret; - make_call_t0(rec->ip, addr, call); + ret = ftrace_rec_update_ops(rec); + if (ret) + return ret; - if (patch_insn_write((void *)rec->ip, call, MCOUNT_INSN_SIZE)) - return -EPERM; + orig_addr = (unsigned long)&ftrace_caller; + distance = addr > orig_addr ? addr - orig_addr : orig_addr - addr; + if (distance > JALR_RANGE) + addr = FTRACE_ADDR; - return 0; + return __ftrace_modify_call(pc, addr, false); } -int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, - unsigned long addr) +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { - unsigned int nops[2] = {RISCV_INSN_NOP4, RISCV_INSN_NOP4}; + u32 nop4 = RISCV_INSN_NOP4; + int ret; - if (patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE)) + ret = ftrace_rec_set_nop_ops(rec); + if (ret) + return ret; + + if (patch_insn_write((void *)rec->ip, &nop4, MCOUNT_NOP4_SIZE)) return -EPERM; return 0; @@ -114,75 +145,71 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, */ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) { - int out; + unsigned long pc = rec->ip - MCOUNT_AUIPC_SIZE; + unsigned int nops[2], offset; + int ret; - mutex_lock(&text_mutex); - out = ftrace_make_nop(mod, rec, MCOUNT_ADDR); - mutex_unlock(&text_mutex); + ret = ftrace_rec_set_nop_ops(rec); + if (ret) + return ret; - return out; -} + offset = (unsigned long) &ftrace_caller - pc; + nops[0] = to_auipc_t0(offset); + nops[1] = RISCV_INSN_NOP4; -int ftrace_update_ftrace_func(ftrace_func_t func) -{ - int ret = __ftrace_modify_call((unsigned long)&ftrace_call, - (unsigned long)func, true, true); + mutex_lock(&text_mutex); + ret = patch_insn_write((void *)pc, nops, 2 * MCOUNT_INSN_SIZE); + mutex_unlock(&text_mutex); return ret; } -struct ftrace_modify_param { - int command; - atomic_t cpu_count; -}; - -static int __ftrace_modify_code(void *data) +ftrace_func_t ftrace_call_dest = ftrace_stub; +int ftrace_update_ftrace_func(ftrace_func_t func) { - struct ftrace_modify_param *param = data; - - if (atomic_inc_return(¶m->cpu_count) == num_online_cpus()) { - ftrace_modify_all_code(param->command); - /* - * Make sure the patching store is effective *before* we - * increment the counter which releases all waiting CPUs - * by using the release variant of atomic increment. The - * release pairs with the call to local_flush_icache_all() - * on the waiting CPU. - */ - atomic_inc_return_release(¶m->cpu_count); - } else { - while (atomic_read(¶m->cpu_count) <= num_online_cpus()) - cpu_relax(); - - local_flush_icache_all(); - } + /* + * When using CALL_OPS, the function to call is associated with the + * call site, and we don't have a global function pointer to update. + */ + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) + return 0; + WRITE_ONCE(ftrace_call_dest, func); + /* + * The data fence ensure that the update to ftrace_call_dest happens + * before the write to function_trace_op later in the generic ftrace. + * If the sequence is not enforced, then an old ftrace_call_dest may + * race loading a new function_trace_op set in ftrace_modify_all_code + */ + smp_wmb(); + /* + * Updating ftrace dpes not take stop_machine path, so irqs should not + * be disabled. + */ + WARN_ON(irqs_disabled()); + smp_call_function(ftrace_sync_ipi, NULL, 1); return 0; } -void arch_ftrace_update_code(int command) +#else /* CONFIG_DYNAMIC_FTRACE */ +unsigned long ftrace_call_adjust(unsigned long addr) { - struct ftrace_modify_param param = { command, ATOMIC_INIT(0) }; - - stop_machine(__ftrace_modify_code, ¶m, cpu_online_mask); + return addr; } -#endif +#endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr) { - unsigned int call[2]; - unsigned long caller = rec->ip; + unsigned long caller = rec->ip - MCOUNT_AUIPC_SIZE; int ret; - make_call_t0(caller, old_addr, call); - ret = ftrace_check_current_call(caller, call); - + ret = ftrace_rec_update_ops(rec); if (ret) return ret; - return __ftrace_modify_call(caller, addr, true, false); + return __ftrace_modify_call(caller, FTRACE_ADDR, true); } #endif @@ -210,7 +237,6 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr, } #ifdef CONFIG_DYNAMIC_FTRACE -#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct ftrace_regs *fregs) { @@ -231,19 +257,5 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, if (!function_graph_enter_regs(old, ip, frame_pointer, parent, fregs)) *parent = return_hooker; } -#else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ -extern void ftrace_graph_call(void); -int ftrace_enable_ftrace_graph_caller(void) -{ - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, true, true); -} - -int ftrace_disable_ftrace_graph_caller(void) -{ - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, false, true); -} -#endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ diff --git a/arch/riscv/kernel/kexec_elf.c b/arch/riscv/kernel/kexec_elf.c new file mode 100644 index 000000000000..f4755d49b89e --- /dev/null +++ b/arch/riscv/kernel/kexec_elf.c @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Load ELF vmlinux file for the kexec_file_load syscall. + * + * Copyright (C) 2021 Huawei Technologies Co, Ltd. + * + * Author: Liao Chang (liaochang1@huawei.com) + * + * Based on kexec-tools' kexec-elf-riscv.c, heavily modified + * for kernel. + */ + +#define pr_fmt(fmt) "kexec_image: " fmt + +#include <linux/elf.h> +#include <linux/kexec.h> +#include <linux/slab.h> +#include <linux/of.h> +#include <linux/libfdt.h> +#include <linux/types.h> +#include <linux/memblock.h> +#include <asm/setup.h> + +static int riscv_kexec_elf_load(struct kimage *image, struct elfhdr *ehdr, + struct kexec_elf_info *elf_info, unsigned long old_pbase, + unsigned long new_pbase) +{ + int i; + int ret = 0; + size_t size; + struct kexec_buf kbuf; + const struct elf_phdr *phdr; + + kbuf.image = image; + + for (i = 0; i < ehdr->e_phnum; i++) { + phdr = &elf_info->proghdrs[i]; + if (phdr->p_type != PT_LOAD) + continue; + + size = phdr->p_filesz; + if (size > phdr->p_memsz) + size = phdr->p_memsz; + + kbuf.buffer = (void *) elf_info->buffer + phdr->p_offset; + kbuf.bufsz = size; + kbuf.buf_align = phdr->p_align; + kbuf.mem = phdr->p_paddr - old_pbase + new_pbase; + kbuf.memsz = phdr->p_memsz; + kbuf.top_down = false; + ret = kexec_add_buffer(&kbuf); + if (ret) + break; + } + + return ret; +} + +/* + * Go through the available phsyical memory regions and find one that hold + * an image of the specified size. + */ +static int elf_find_pbase(struct kimage *image, unsigned long kernel_len, + struct elfhdr *ehdr, struct kexec_elf_info *elf_info, + unsigned long *old_pbase, unsigned long *new_pbase) +{ + int i; + int ret; + struct kexec_buf kbuf; + const struct elf_phdr *phdr; + unsigned long lowest_paddr = ULONG_MAX; + unsigned long lowest_vaddr = ULONG_MAX; + + for (i = 0; i < ehdr->e_phnum; i++) { + phdr = &elf_info->proghdrs[i]; + if (phdr->p_type != PT_LOAD) + continue; + + if (lowest_paddr > phdr->p_paddr) + lowest_paddr = phdr->p_paddr; + + if (lowest_vaddr > phdr->p_vaddr) + lowest_vaddr = phdr->p_vaddr; + } + + kbuf.image = image; + kbuf.buf_min = lowest_paddr; + kbuf.buf_max = ULONG_MAX; + + /* + * Current riscv boot protocol requires 2MB alignment for + * RV64 and 4MB alignment for RV32 + * + */ + kbuf.buf_align = PMD_SIZE; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + kbuf.memsz = ALIGN(kernel_len, PAGE_SIZE); + kbuf.top_down = false; + ret = arch_kexec_locate_mem_hole(&kbuf); + if (!ret) { + *old_pbase = lowest_paddr; + *new_pbase = kbuf.mem; + image->start = ehdr->e_entry - lowest_vaddr + kbuf.mem; + } + return ret; +} + +static void *elf_kexec_load(struct kimage *image, char *kernel_buf, + unsigned long kernel_len, char *initrd, + unsigned long initrd_len, char *cmdline, + unsigned long cmdline_len) +{ + int ret; + unsigned long old_kernel_pbase = ULONG_MAX; + unsigned long new_kernel_pbase = 0UL; + struct elfhdr ehdr; + struct kexec_elf_info elf_info; + + ret = kexec_build_elf_info(kernel_buf, kernel_len, &ehdr, &elf_info); + if (ret) + return ERR_PTR(ret); + + ret = elf_find_pbase(image, kernel_len, &ehdr, &elf_info, + &old_kernel_pbase, &new_kernel_pbase); + if (ret) + goto out; + + /* Add the kernel binary to the image */ + ret = riscv_kexec_elf_load(image, &ehdr, &elf_info, + old_kernel_pbase, new_kernel_pbase); + if (ret) + goto out; + + ret = load_extra_segments(image, image->start, kernel_len, + initrd, initrd_len, cmdline, cmdline_len); +out: + kexec_free_elf_info(&elf_info); + return ret ? ERR_PTR(ret) : NULL; +} + +const struct kexec_file_ops elf_kexec_ops = { + .probe = kexec_elf_probe, + .load = elf_kexec_load, +}; diff --git a/arch/riscv/kernel/kexec_image.c b/arch/riscv/kernel/kexec_image.c new file mode 100644 index 000000000000..26a81774a78a --- /dev/null +++ b/arch/riscv/kernel/kexec_image.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * RISC-V Kexec image loader + * + */ + +#define pr_fmt(fmt) "kexec_file(Image): " fmt + +#include <linux/err.h> +#include <linux/errno.h> +#include <linux/kernel.h> +#include <linux/kexec.h> +#include <linux/pe.h> +#include <linux/string.h> +#include <asm/byteorder.h> +#include <asm/image.h> + +static int image_probe(const char *kernel_buf, unsigned long kernel_len) +{ + const struct riscv_image_header *h = (const struct riscv_image_header *)kernel_buf; + + if (!h || kernel_len < sizeof(*h)) + return -EINVAL; + + /* According to Documentation/riscv/boot-image-header.rst, + * use "magic2" field to check when version >= 0.2. + */ + + if (h->version >= RISCV_HEADER_VERSION && + memcmp(&h->magic2, RISCV_IMAGE_MAGIC2, sizeof(h->magic2))) + return -EINVAL; + + return 0; +} + +static void *image_load(struct kimage *image, + char *kernel, unsigned long kernel_len, + char *initrd, unsigned long initrd_len, + char *cmdline, unsigned long cmdline_len) +{ + struct riscv_image_header *h; + u64 flags; + bool be_image, be_kernel; + struct kexec_buf kbuf; + int ret; + + /* Check Image header */ + h = (struct riscv_image_header *)kernel; + if (!h->image_size) { + ret = -EINVAL; + goto out; + } + + /* Check endianness */ + flags = le64_to_cpu(h->flags); + be_image = riscv_image_flag_field(flags, RISCV_IMAGE_FLAG_BE); + be_kernel = IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); + if (be_image != be_kernel) { + ret = -EINVAL; + goto out; + } + + /* Load the kernel image */ + kbuf.image = image; + kbuf.buf_min = 0; + kbuf.buf_max = ULONG_MAX; + kbuf.top_down = false; + + kbuf.buffer = kernel; + kbuf.bufsz = kernel_len; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + kbuf.memsz = le64_to_cpu(h->image_size); + kbuf.buf_align = le64_to_cpu(h->text_offset); + + ret = kexec_add_buffer(&kbuf); + if (ret) { + pr_err("Error add kernel image ret=%d\n", ret); + goto out; + } + + image->start = kbuf.mem; + + pr_info("Loaded kernel at 0x%lx bufsz=0x%lx memsz=0x%lx\n", + kbuf.mem, kbuf.bufsz, kbuf.memsz); + + ret = load_extra_segments(image, kbuf.mem, kbuf.memsz, + initrd, initrd_len, cmdline, cmdline_len); + +out: + return ret ? ERR_PTR(ret) : NULL; +} + +const struct kexec_file_ops image_kexec_ops = { + .probe = image_probe, + .load = image_load, +}; diff --git a/arch/riscv/kernel/machine_kexec_file.c b/arch/riscv/kernel/machine_kexec_file.c index b0bf8c1722c0..e36104af2e24 100644 --- a/arch/riscv/kernel/machine_kexec_file.c +++ b/arch/riscv/kernel/machine_kexec_file.c @@ -7,8 +7,369 @@ * Author: Liao Chang (liaochang1@huawei.com) */ #include <linux/kexec.h> +#include <linux/elf.h> +#include <linux/slab.h> +#include <linux/of.h> +#include <linux/libfdt.h> +#include <linux/types.h> +#include <linux/memblock.h> +#include <linux/vmalloc.h> +#include <asm/setup.h> const struct kexec_file_ops * const kexec_file_loaders[] = { &elf_kexec_ops, + &image_kexec_ops, NULL }; + +int arch_kimage_file_post_load_cleanup(struct kimage *image) +{ + kvfree(image->arch.fdt); + image->arch.fdt = NULL; + + vfree(image->elf_headers); + image->elf_headers = NULL; + image->elf_headers_sz = 0; + + return kexec_image_post_load_cleanup_default(image); +} + +#ifdef CONFIG_CRASH_DUMP +static int get_nr_ram_ranges_callback(struct resource *res, void *arg) +{ + unsigned int *nr_ranges = arg; + + (*nr_ranges)++; + return 0; +} + +static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) +{ + struct crash_mem *cmem = arg; + + cmem->ranges[cmem->nr_ranges].start = res->start; + cmem->ranges[cmem->nr_ranges].end = res->end; + cmem->nr_ranges++; + + return 0; +} + +static int prepare_elf_headers(void **addr, unsigned long *sz) +{ + struct crash_mem *cmem; + unsigned int nr_ranges; + int ret; + + nr_ranges = 1; /* For exclusion of crashkernel region */ + walk_system_ram_res(0, -1, &nr_ranges, get_nr_ram_ranges_callback); + + cmem = kmalloc(struct_size(cmem, ranges, nr_ranges), GFP_KERNEL); + if (!cmem) + return -ENOMEM; + + cmem->max_nr_ranges = nr_ranges; + cmem->nr_ranges = 0; + ret = walk_system_ram_res(0, -1, cmem, prepare_elf64_ram_headers_callback); + if (ret) + goto out; + + /* Exclude crashkernel region */ + ret = crash_exclude_mem_range(cmem, crashk_res.start, crashk_res.end); + if (!ret) + ret = crash_prepare_elf64_headers(cmem, true, addr, sz); + +out: + kfree(cmem); + return ret; +} + +static char *setup_kdump_cmdline(struct kimage *image, char *cmdline, + unsigned long cmdline_len) +{ + int elfcorehdr_strlen; + char *cmdline_ptr; + + cmdline_ptr = kzalloc(COMMAND_LINE_SIZE, GFP_KERNEL); + if (!cmdline_ptr) + return NULL; + + elfcorehdr_strlen = sprintf(cmdline_ptr, "elfcorehdr=0x%lx ", + image->elf_load_addr); + + if (elfcorehdr_strlen + cmdline_len > COMMAND_LINE_SIZE) { + pr_err("Appending elfcorehdr=<addr> exceeds cmdline size\n"); + kfree(cmdline_ptr); + return NULL; + } + + memcpy(cmdline_ptr + elfcorehdr_strlen, cmdline, cmdline_len); + /* Ensure it's nul terminated */ + cmdline_ptr[COMMAND_LINE_SIZE - 1] = '\0'; + return cmdline_ptr; +} +#endif + +#define RV_X(x, s, n) (((x) >> (s)) & ((1 << (n)) - 1)) +#define RISCV_IMM_BITS 12 +#define RISCV_IMM_REACH (1LL << RISCV_IMM_BITS) +#define RISCV_CONST_HIGH_PART(x) \ + (((x) + (RISCV_IMM_REACH >> 1)) & ~(RISCV_IMM_REACH - 1)) +#define RISCV_CONST_LOW_PART(x) ((x) - RISCV_CONST_HIGH_PART(x)) + +#define ENCODE_ITYPE_IMM(x) \ + (RV_X(x, 0, 12) << 20) +#define ENCODE_BTYPE_IMM(x) \ + ((RV_X(x, 1, 4) << 8) | (RV_X(x, 5, 6) << 25) | \ + (RV_X(x, 11, 1) << 7) | (RV_X(x, 12, 1) << 31)) +#define ENCODE_UTYPE_IMM(x) \ + (RV_X(x, 12, 20) << 12) +#define ENCODE_JTYPE_IMM(x) \ + ((RV_X(x, 1, 10) << 21) | (RV_X(x, 11, 1) << 20) | \ + (RV_X(x, 12, 8) << 12) | (RV_X(x, 20, 1) << 31)) +#define ENCODE_CBTYPE_IMM(x) \ + ((RV_X(x, 1, 2) << 3) | (RV_X(x, 3, 2) << 10) | (RV_X(x, 5, 1) << 2) | \ + (RV_X(x, 6, 2) << 5) | (RV_X(x, 8, 1) << 12)) +#define ENCODE_CJTYPE_IMM(x) \ + ((RV_X(x, 1, 3) << 3) | (RV_X(x, 4, 1) << 11) | (RV_X(x, 5, 1) << 2) | \ + (RV_X(x, 6, 1) << 7) | (RV_X(x, 7, 1) << 6) | (RV_X(x, 8, 2) << 9) | \ + (RV_X(x, 10, 1) << 8) | (RV_X(x, 11, 1) << 12)) +#define ENCODE_UJTYPE_IMM(x) \ + (ENCODE_UTYPE_IMM(RISCV_CONST_HIGH_PART(x)) | \ + (ENCODE_ITYPE_IMM(RISCV_CONST_LOW_PART(x)) << 32)) +#define ENCODE_UITYPE_IMM(x) \ + (ENCODE_UTYPE_IMM(x) | (ENCODE_ITYPE_IMM(x) << 32)) + +#define CLEAN_IMM(type, x) \ + ((~ENCODE_##type##_IMM((uint64_t)(-1))) & (x)) + +int arch_kexec_apply_relocations_add(struct purgatory_info *pi, + Elf_Shdr *section, + const Elf_Shdr *relsec, + const Elf_Shdr *symtab) +{ + const char *strtab, *name, *shstrtab; + const Elf_Shdr *sechdrs; + Elf64_Rela *relas; + int i, r_type; + + /* String & section header string table */ + sechdrs = (void *)pi->ehdr + pi->ehdr->e_shoff; + strtab = (char *)pi->ehdr + sechdrs[symtab->sh_link].sh_offset; + shstrtab = (char *)pi->ehdr + sechdrs[pi->ehdr->e_shstrndx].sh_offset; + + relas = (void *)pi->ehdr + relsec->sh_offset; + + for (i = 0; i < relsec->sh_size / sizeof(*relas); i++) { + const Elf_Sym *sym; /* symbol to relocate */ + unsigned long addr; /* final location after relocation */ + unsigned long val; /* relocated symbol value */ + unsigned long sec_base; /* relocated symbol value */ + void *loc; /* tmp location to modify */ + + sym = (void *)pi->ehdr + symtab->sh_offset; + sym += ELF64_R_SYM(relas[i].r_info); + + if (sym->st_name) + name = strtab + sym->st_name; + else + name = shstrtab + sechdrs[sym->st_shndx].sh_name; + + loc = pi->purgatory_buf; + loc += section->sh_offset; + loc += relas[i].r_offset; + + if (sym->st_shndx == SHN_ABS) + sec_base = 0; + else if (sym->st_shndx >= pi->ehdr->e_shnum) { + pr_err("Invalid section %d for symbol %s\n", + sym->st_shndx, name); + return -ENOEXEC; + } else + sec_base = pi->sechdrs[sym->st_shndx].sh_addr; + + val = sym->st_value; + val += sec_base; + val += relas[i].r_addend; + + addr = section->sh_addr + relas[i].r_offset; + + r_type = ELF64_R_TYPE(relas[i].r_info); + + switch (r_type) { + case R_RISCV_BRANCH: + *(u32 *)loc = CLEAN_IMM(BTYPE, *(u32 *)loc) | + ENCODE_BTYPE_IMM(val - addr); + break; + case R_RISCV_JAL: + *(u32 *)loc = CLEAN_IMM(JTYPE, *(u32 *)loc) | + ENCODE_JTYPE_IMM(val - addr); + break; + /* + * With no R_RISCV_PCREL_LO12_S, R_RISCV_PCREL_LO12_I + * sym is expected to be next to R_RISCV_PCREL_HI20 + * in purgatory relsec. Handle it like R_RISCV_CALL + * sym, instead of searching the whole relsec. + */ + case R_RISCV_PCREL_HI20: + case R_RISCV_CALL_PLT: + case R_RISCV_CALL: + *(u64 *)loc = CLEAN_IMM(UITYPE, *(u64 *)loc) | + ENCODE_UJTYPE_IMM(val - addr); + break; + case R_RISCV_RVC_BRANCH: + *(u32 *)loc = CLEAN_IMM(CBTYPE, *(u32 *)loc) | + ENCODE_CBTYPE_IMM(val - addr); + break; + case R_RISCV_RVC_JUMP: + *(u32 *)loc = CLEAN_IMM(CJTYPE, *(u32 *)loc) | + ENCODE_CJTYPE_IMM(val - addr); + break; + case R_RISCV_ADD16: + *(u16 *)loc += val; + break; + case R_RISCV_SUB16: + *(u16 *)loc -= val; + break; + case R_RISCV_ADD32: + *(u32 *)loc += val; + break; + case R_RISCV_SUB32: + *(u32 *)loc -= val; + break; + /* It has been applied by R_RISCV_PCREL_HI20 sym */ + case R_RISCV_PCREL_LO12_I: + case R_RISCV_ALIGN: + case R_RISCV_RELAX: + break; + case R_RISCV_64: + *(u64 *)loc = val; + break; + default: + pr_err("Unknown rela relocation: %d\n", r_type); + return -ENOEXEC; + } + } + return 0; +} + + +int load_extra_segments(struct kimage *image, unsigned long kernel_start, + unsigned long kernel_len, char *initrd, + unsigned long initrd_len, char *cmdline, + unsigned long cmdline_len) +{ + int ret; + void *fdt; + unsigned long initrd_pbase = 0UL; + struct kexec_buf kbuf; + char *modified_cmdline = NULL; + + kbuf.image = image; + kbuf.buf_min = kernel_start + kernel_len; + kbuf.buf_max = ULONG_MAX; + +#ifdef CONFIG_CRASH_DUMP + /* Add elfcorehdr */ + if (image->type == KEXEC_TYPE_CRASH) { + void *headers; + unsigned long headers_sz; + ret = prepare_elf_headers(&headers, &headers_sz); + if (ret) { + pr_err("Preparing elf core header failed\n"); + goto out; + } + + kbuf.buffer = headers; + kbuf.bufsz = headers_sz; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + kbuf.memsz = headers_sz; + kbuf.buf_align = ELF_CORE_HEADER_ALIGN; + kbuf.top_down = true; + + ret = kexec_add_buffer(&kbuf); + if (ret) { + vfree(headers); + goto out; + } + image->elf_headers = headers; + image->elf_load_addr = kbuf.mem; + image->elf_headers_sz = headers_sz; + + kexec_dprintk("Loaded elf core header at 0x%lx bufsz=0x%lx memsz=0x%lx\n", + image->elf_load_addr, kbuf.bufsz, kbuf.memsz); + + /* Setup cmdline for kdump kernel case */ + modified_cmdline = setup_kdump_cmdline(image, cmdline, + cmdline_len); + if (!modified_cmdline) { + pr_err("Setting up cmdline for kdump kernel failed\n"); + ret = -EINVAL; + goto out; + } + cmdline = modified_cmdline; + } +#endif + +#ifdef CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY + /* Add purgatory to the image */ + kbuf.top_down = true; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + ret = kexec_load_purgatory(image, &kbuf); + if (ret) { + pr_err("Error loading purgatory ret=%d\n", ret); + goto out; + } + kexec_dprintk("Loaded purgatory at 0x%lx\n", kbuf.mem); + + ret = kexec_purgatory_get_set_symbol(image, "riscv_kernel_entry", + &kernel_start, + sizeof(kernel_start), 0); + if (ret) + pr_err("Error update purgatory ret=%d\n", ret); +#endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */ + + /* Add the initrd to the image */ + if (initrd != NULL) { + kbuf.buffer = initrd; + kbuf.bufsz = kbuf.memsz = initrd_len; + kbuf.buf_align = PAGE_SIZE; + kbuf.top_down = true; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + ret = kexec_add_buffer(&kbuf); + if (ret) + goto out; + initrd_pbase = kbuf.mem; + kexec_dprintk("Loaded initrd at 0x%lx\n", initrd_pbase); + } + + /* Add the DTB to the image */ + fdt = of_kexec_alloc_and_setup_fdt(image, initrd_pbase, + initrd_len, cmdline, 0); + if (!fdt) { + pr_err("Error setting up the new device tree.\n"); + ret = -EINVAL; + goto out; + } + + fdt_pack(fdt); + kbuf.buffer = fdt; + kbuf.bufsz = kbuf.memsz = fdt_totalsize(fdt); + kbuf.buf_align = PAGE_SIZE; + kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; + kbuf.top_down = true; + ret = kexec_add_buffer(&kbuf); + if (ret) { + pr_err("Error add DTB kbuf ret=%d\n", ret); + goto out_free_fdt; + } + /* Cache the fdt buffer address for memory cleanup */ + image->arch.fdt = fdt; + kexec_dprintk("Loaded device tree at 0x%lx\n", kbuf.mem); + goto out; + +out_free_fdt: + kvfree(fdt); +out: + kfree(modified_cmdline); + return ret; +} diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index 745dd4c4a69c..48f6c4f7dca0 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -13,7 +13,6 @@ .text -#define FENTRY_RA_OFFSET 8 #define ABI_SIZE_ON_STACK 80 #define ABI_A0 0 #define ABI_A1 8 @@ -56,16 +55,13 @@ addi sp, sp, ABI_SIZE_ON_STACK .endm -#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS - /** * SAVE_ABI_REGS - save regs against the ftrace_regs struct * * After the stack is established, * * 0(sp) stores the PC of the traced function which can be accessed -* by &(fregs)->epc in tracing function. Note that the real -* function entry address should be computed with -FENTRY_RA_OFFSET. +* by &(fregs)->epc in tracing function. * * 8(sp) stores the function return address (i.e. parent IP) that * can be accessed by &(fregs)->ra in tracing function. @@ -86,17 +82,20 @@ * +++++++++ **/ .macro SAVE_ABI_REGS - mv t4, sp // Save original SP in T4 addi sp, sp, -FREGS_SIZE_ON_STACK - REG_S t0, FREGS_EPC(sp) REG_S x1, FREGS_RA(sp) - REG_S t4, FREGS_SP(sp) // Put original SP on stack #ifdef HAVE_FUNCTION_GRAPH_FP_TEST REG_S x8, FREGS_S0(sp) #endif REG_S x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_S x7, FREGS_T2(sp) + REG_S x28, FREGS_T3(sp) + REG_S x29, FREGS_T4(sp) + REG_S x30, FREGS_T5(sp) + REG_S x31, FREGS_T6(sp) +#endif // save the arguments REG_S x10, FREGS_A0(sp) REG_S x11, FREGS_A1(sp) @@ -106,16 +105,25 @@ REG_S x15, FREGS_A5(sp) REG_S x16, FREGS_A6(sp) REG_S x17, FREGS_A7(sp) + mv a0, sp + addi a0, a0, FREGS_SIZE_ON_STACK + REG_S a0, FREGS_SP(sp) // Put original SP on stack .endm - .macro RESTORE_ABI_REGS, all=0 + .macro RESTORE_ABI_REGS REG_L t0, FREGS_EPC(sp) REG_L x1, FREGS_RA(sp) #ifdef HAVE_FUNCTION_GRAPH_FP_TEST REG_L x8, FREGS_S0(sp) #endif REG_L x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_L x7, FREGS_T2(sp) + REG_L x28, FREGS_T3(sp) + REG_L x29, FREGS_T4(sp) + REG_L x30, FREGS_T5(sp) + REG_L x31, FREGS_T6(sp) +#endif // restore the arguments REG_L x10, FREGS_A0(sp) REG_L x11, FREGS_A1(sp) @@ -130,60 +138,71 @@ .endm .macro PREPARE_ARGS - addi a0, t0, -FENTRY_RA_OFFSET + addi a0, t0, -MCOUNT_JALR_SIZE // ip (callsite's jalr insn) +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS + mv a1, ra // parent_ip + REG_L a2, -16(t0) // op + REG_L ra, FTRACE_OPS_FUNC(a2) // op->func +#else la a1, function_trace_op - REG_L a2, 0(a1) - mv a1, ra - mv a3, sp + REG_L a2, 0(a1) // op + mv a1, ra // parent_ip +#endif + mv a3, sp // regs .endm -#endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ - -#ifndef CONFIG_DYNAMIC_FTRACE_WITH_ARGS SYM_FUNC_START(ftrace_caller) - SAVE_ABI - - addi a0, t0, -FENTRY_RA_OFFSET - la a1, function_trace_op - REG_L a2, 0(a1) - mv a1, ra - mv a3, sp +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS + /* + * When CALL_OPS is enabled (2 or 4) nops [8B] are placed before the + * function entry, these are later overwritten with the pointer to the + * associated struct ftrace_ops. + * + * -8: &ftrace_ops of the associated tracer function. + *<ftrace enable>: + * 0: auipc t0/ra, 0x? + * 4: jalr t0/ra, ?(t0/ra) + * + * -8: &ftrace_nop_ops + *<ftrace disable>: + * 0: nop + * 4: nop + * + * t0 is set to ip+8 after the jalr is executed at the callsite, + * so we find the associated op at t0-16. + */ + REG_L t1, -16(t0) // op Should be SZ_REG instead of 16 -SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub - -#ifdef CONFIG_FUNCTION_GRAPH_TRACER - addi a0, sp, ABI_RA - REG_L a1, ABI_T0(sp) - addi a1, a1, -FENTRY_RA_OFFSET -#ifdef HAVE_FUNCTION_GRAPH_FP_TEST - mv a2, s0 +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + /* + * If the op has a direct call, handle it immediately without + * saving/restoring registers. + */ + REG_L t1, FTRACE_OPS_DIRECT_CALL(t1) + bnez t1, ftrace_caller_direct #endif -SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - call ftrace_stub #endif - RESTORE_ABI - jr t0 -SYM_FUNC_END(ftrace_caller) - -#else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ -SYM_FUNC_START(ftrace_caller) - mv t1, zero SAVE_ABI_REGS PREPARE_ARGS +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS + jalr ra +#else SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub - + REG_L ra, ftrace_call_dest + jalr ra, 0(ra) +#endif RESTORE_ABI_REGS - bnez t1, .Ldirect +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + bnez t1, ftrace_caller_direct +#endif jr t0 -.Ldirect: +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS +SYM_INNER_LABEL(ftrace_caller_direct, SYM_L_LOCAL) jr t1 +#endif SYM_FUNC_END(ftrace_caller) -#endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ - #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS SYM_CODE_START(ftrace_stub_direct_tramp) jr t0 diff --git a/arch/riscv/kernel/module-sections.c b/arch/riscv/kernel/module-sections.c index 91d0b355ceef..75551ac6504c 100644 --- a/arch/riscv/kernel/module-sections.c +++ b/arch/riscv/kernel/module-sections.c @@ -9,6 +9,7 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/moduleloader.h> +#include <linux/sort.h> unsigned long module_emit_got_entry(struct module *mod, unsigned long val) { @@ -55,44 +56,70 @@ unsigned long module_emit_plt_entry(struct module *mod, unsigned long val) return (unsigned long)&plt[i]; } -static int is_rela_equal(const Elf_Rela *x, const Elf_Rela *y) +#define cmp_3way(a, b) ((a) < (b) ? -1 : (a) > (b)) + +static int cmp_rela(const void *a, const void *b) { - return x->r_info == y->r_info && x->r_addend == y->r_addend; + const Elf_Rela *x = a, *y = b; + int i; + + /* sort by type, symbol index and addend */ + i = cmp_3way(x->r_info, y->r_info); + if (i == 0) + i = cmp_3way(x->r_addend, y->r_addend); + return i; } static bool duplicate_rela(const Elf_Rela *rela, int idx) { - int i; - for (i = 0; i < idx; i++) { - if (is_rela_equal(&rela[i], &rela[idx])) - return true; - } - return false; + /* + * Entries are sorted by type, symbol index and addend. That means + * that, if a duplicate entry exists, it must be in the preceding slot. + */ + return idx > 0 && cmp_rela(rela + idx, rela + idx - 1) == 0; } -static void count_max_entries(Elf_Rela *relas, int num, +static void count_max_entries(const Elf_Rela *relas, size_t num, unsigned int *plts, unsigned int *gots) { - for (int i = 0; i < num; i++) { + for (size_t i = 0; i < num; i++) { + if (duplicate_rela(relas, i)) + continue; + switch (ELF_R_TYPE(relas[i].r_info)) { case R_RISCV_CALL_PLT: case R_RISCV_PLT32: - if (!duplicate_rela(relas, i)) - (*plts)++; + (*plts)++; break; case R_RISCV_GOT_HI20: - if (!duplicate_rela(relas, i)) - (*gots)++; + (*gots)++; break; + default: + unreachable(); } } } +static bool rela_needs_plt_got_entry(const Elf_Rela *rela) +{ + switch (ELF_R_TYPE(rela->r_info)) { + case R_RISCV_CALL_PLT: + case R_RISCV_GOT_HI20: + case R_RISCV_PLT32: + return true; + default: + return false; + } +} + int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, char *secstrings, struct module *mod) { + size_t num_scratch_relas = 0; unsigned int num_plts = 0; unsigned int num_gots = 0; + Elf_Rela *scratch = NULL; + size_t scratch_size = 0; int i; /* @@ -122,9 +149,10 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, /* Calculate the maxinum number of entries */ for (i = 0; i < ehdr->e_shnum; i++) { + size_t num_relas = sechdrs[i].sh_size / sizeof(Elf_Rela); Elf_Rela *relas = (void *)ehdr + sechdrs[i].sh_offset; - int num_rela = sechdrs[i].sh_size / sizeof(Elf_Rela); Elf_Shdr *dst_sec = sechdrs + sechdrs[i].sh_info; + size_t scratch_size_needed; if (sechdrs[i].sh_type != SHT_RELA) continue; @@ -133,7 +161,28 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, if (!(dst_sec->sh_flags & SHF_EXECINSTR)) continue; - count_max_entries(relas, num_rela, &num_plts, &num_gots); + /* + * apply_relocate_add() relies on HI20 and LO12 relocation pairs being + * close together, so sort a copy of the section to avoid interfering. + */ + scratch_size_needed = (num_scratch_relas + num_relas) * sizeof(*scratch); + if (scratch_size_needed > scratch_size) { + scratch_size = scratch_size_needed; + scratch = kvrealloc(scratch, scratch_size, GFP_KERNEL); + if (!scratch) + return -ENOMEM; + } + + for (size_t j = 0; j < num_relas; j++) + if (rela_needs_plt_got_entry(&relas[j])) + scratch[num_scratch_relas++] = relas[j]; + } + + if (scratch) { + /* sort the accumulated PLT/GOT relocations so duplicates are adjacent */ + sort(scratch, num_scratch_relas, sizeof(*scratch), cmp_rela, NULL); + count_max_entries(scratch, num_scratch_relas, &num_plts, &num_gots); + kvfree(scratch); } mod->arch.plt.shdr->sh_type = SHT_NOBITS; diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index bbf7ec6a75c0..a0a40889d79a 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -60,7 +60,7 @@ int get_unalign_ctl(struct task_struct *tsk, unsigned long adr) if (!unaligned_ctl_available()) return -EINVAL; - return put_user(tsk->thread.align_ctl, (unsigned long __user *)adr); + return put_user(tsk->thread.align_ctl, (unsigned int __user *)adr); } void __show_regs(struct pt_regs *regs) diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c index 1989b8cade1b..53836a9235e3 100644 --- a/arch/riscv/kernel/sbi.c +++ b/arch/riscv/kernel/sbi.c @@ -299,6 +299,76 @@ static int __sbi_rfence_v02(int fid, const struct cpumask *cpu_mask, return 0; } +static bool sbi_fwft_supported; + +struct fwft_set_req { + u32 feature; + unsigned long value; + unsigned long flags; + atomic_t error; +}; + +static void cpu_sbi_fwft_set(void *arg) +{ + struct fwft_set_req *req = arg; + int ret; + + ret = sbi_fwft_set(req->feature, req->value, req->flags); + if (ret) + atomic_set(&req->error, ret); +} + +/** + * sbi_fwft_set() - Set a feature on the local hart + * @feature: The feature ID to be set + * @value: The feature value to be set + * @flags: FWFT feature set flags + * + * Return: 0 on success, appropriate linux error code otherwise. + */ +int sbi_fwft_set(u32 feature, unsigned long value, unsigned long flags) +{ + struct sbiret ret; + + if (!sbi_fwft_supported) + return -EOPNOTSUPP; + + ret = sbi_ecall(SBI_EXT_FWFT, SBI_EXT_FWFT_SET, + feature, value, flags, 0, 0, 0); + + return sbi_err_map_linux_errno(ret.error); +} + +/** + * sbi_fwft_set_cpumask() - Set a feature for the specified cpumask + * @mask: CPU mask of cpus that need the feature to be set + * @feature: The feature ID to be set + * @value: The feature value to be set + * @flags: FWFT feature set flags + * + * Return: 0 on success, appropriate linux error code otherwise. + */ +int sbi_fwft_set_cpumask(const cpumask_t *mask, u32 feature, + unsigned long value, unsigned long flags) +{ + struct fwft_set_req req = { + .feature = feature, + .value = value, + .flags = flags, + .error = ATOMIC_INIT(0), + }; + + if (!sbi_fwft_supported) + return -EOPNOTSUPP; + + if (feature & SBI_FWFT_GLOBAL_FEATURE_BIT) + return -EINVAL; + + on_each_cpu_mask(mask, cpu_sbi_fwft_set, &req, 1); + + return atomic_read(&req.error); +} + /** * sbi_set_timer() - Program the timer for next timer event. * @stime_value: The value after which next timer event should fire. @@ -609,7 +679,7 @@ void __init sbi_init(void) } else { __sbi_rfence = __sbi_rfence_v01; } - if ((sbi_spec_version >= sbi_mk_version(0, 3)) && + if (sbi_spec_version >= sbi_mk_version(0, 3) && sbi_probe_extension(SBI_EXT_SRST)) { pr_info("SBI SRST extension detected\n"); pm_power_off = sbi_srst_power_off; @@ -617,11 +687,16 @@ void __init sbi_init(void) sbi_srst_reboot_nb.priority = 192; register_restart_handler(&sbi_srst_reboot_nb); } - if ((sbi_spec_version >= sbi_mk_version(2, 0)) && - (sbi_probe_extension(SBI_EXT_DBCN) > 0)) { + if (sbi_spec_version >= sbi_mk_version(2, 0) && + sbi_probe_extension(SBI_EXT_DBCN) > 0) { pr_info("SBI DBCN extension detected\n"); sbi_debug_console_available = true; } + if (sbi_spec_version >= sbi_mk_version(3, 0) && + sbi_probe_extension(SBI_EXT_FWFT)) { + pr_info("SBI FWFT extension detected\n"); + sbi_fwft_supported = true; + } } else { __sbi_set_timer = __sbi_set_timer_v01; __sbi_send_ipi = __sbi_send_ipi_v01; diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index 249aec8594a9..0b170e18a2be 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -15,6 +15,7 @@ #include <asm/uaccess.h> #include <asm/unistd.h> #include <asm/vector.h> +#include <asm/vendor_extensions/sifive_hwprobe.h> #include <asm/vendor_extensions/thead_hwprobe.h> #include <vdso/vsyscall.h> @@ -96,6 +97,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair, * presence in the hart_isa bitmap, are made. */ EXT_KEY(ZAAMO); + EXT_KEY(ZABHA); EXT_KEY(ZACAS); EXT_KEY(ZALRSC); EXT_KEY(ZAWRS); @@ -300,6 +302,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, pair->value = riscv_timebase; break; + case RISCV_HWPROBE_KEY_VENDOR_EXT_SIFIVE_0: + hwprobe_isa_vendor_ext_sifive_0(pair, cpus); + break; + case RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0: hwprobe_isa_vendor_ext_thead_0(pair, cpus); break; diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index 77c788660223..dd8e4af6583f 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include <asm/entry-common.h> #include <asm/hwprobe.h> #include <asm/cpufeature.h> +#include <asm/sbi.h> #include <asm/vector.h> #define INSN_MATCH_LB 0x3 @@ -368,9 +369,7 @@ static int handle_scalar_misaligned_load(struct pt_regs *regs) perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, addr); -#ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS *this_cpu_ptr(&misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED; -#endif if (!unaligned_enabled) return -1; @@ -455,7 +454,7 @@ static int handle_scalar_misaligned_load(struct pt_regs *regs) val.data_u64 = 0; if (user_mode(regs)) { - if (copy_from_user(&val, (u8 __user *)addr, len)) + if (copy_from_user_nofault(&val, (u8 __user *)addr, len)) return -1; } else { memcpy(&val, (u8 *)addr, len); @@ -556,7 +555,7 @@ static int handle_scalar_misaligned_store(struct pt_regs *regs) return -EOPNOTSUPP; if (user_mode(regs)) { - if (copy_to_user((u8 __user *)addr, &val, len)) + if (copy_to_user_nofault((u8 __user *)addr, &val, len)) return -1; } else { memcpy((u8 *)addr, &val, len); @@ -626,6 +625,10 @@ bool __init check_vector_unaligned_access_emulated_all_cpus(void) { int cpu; + /* + * While being documented as very slow, schedule_on_each_cpu() is used since + * kernel_vector_begin() expects irqs to be enabled or it will panic() + */ schedule_on_each_cpu(check_vector_unaligned_access_emulated); for_each_online_cpu(cpu) @@ -642,11 +645,23 @@ bool __init check_vector_unaligned_access_emulated_all_cpus(void) } #endif +static bool all_cpus_unaligned_scalar_access_emulated(void) +{ + int cpu; + + for_each_online_cpu(cpu) + if (per_cpu(misaligned_access_speed, cpu) != + RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED) + return false; + + return true; +} + #ifdef CONFIG_RISCV_SCALAR_MISALIGNED static bool unaligned_ctl __read_mostly; -void check_unaligned_access_emulated(struct work_struct *work __always_unused) +static void check_unaligned_access_emulated(void *arg __always_unused) { int cpu = smp_processor_id(); long *mas_ptr = per_cpu_ptr(&misaligned_access_speed, cpu); @@ -657,6 +672,13 @@ void check_unaligned_access_emulated(struct work_struct *work __always_unused) __asm__ __volatile__ ( " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); +} + +static int cpu_online_check_unaligned_access_emulated(unsigned int cpu) +{ + long *mas_ptr = per_cpu_ptr(&misaligned_access_speed, cpu); + + check_unaligned_access_emulated(NULL); /* * If unaligned_ctl is already set, this means that we detected that all @@ -665,26 +687,23 @@ void check_unaligned_access_emulated(struct work_struct *work __always_unused) */ if (unlikely(unaligned_ctl && (*mas_ptr != RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED))) { pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)\n"); - while (true) - cpu_relax(); + return -EINVAL; } + + return 0; } bool __init check_unaligned_access_emulated_all_cpus(void) { - int cpu; - /* * We can only support PR_UNALIGN controls if all CPUs have misaligned * accesses emulated since tasks requesting such control can run on any * CPU. */ - schedule_on_each_cpu(check_unaligned_access_emulated); + on_each_cpu(check_unaligned_access_emulated, NULL, 1); - for_each_online_cpu(cpu) - if (per_cpu(misaligned_access_speed, cpu) - != RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED) - return false; + if (!all_cpus_unaligned_scalar_access_emulated()) + return false; unaligned_ctl = true; return true; @@ -699,4 +718,73 @@ bool __init check_unaligned_access_emulated_all_cpus(void) { return false; } +static int cpu_online_check_unaligned_access_emulated(unsigned int cpu) +{ + return 0; +} +#endif + +static bool misaligned_traps_delegated; + +#ifdef CONFIG_RISCV_SBI + +static int cpu_online_sbi_unaligned_setup(unsigned int cpu) +{ + if (sbi_fwft_set(SBI_FWFT_MISALIGNED_EXC_DELEG, 1, 0) && + misaligned_traps_delegated) { + pr_crit("Misaligned trap delegation non homogeneous (expected delegated)"); + return -EINVAL; + } + + return 0; +} + +void __init unaligned_access_init(void) +{ + int ret; + + ret = sbi_fwft_set_online_cpus(SBI_FWFT_MISALIGNED_EXC_DELEG, 1, 0); + if (ret) + return; + + misaligned_traps_delegated = true; + pr_info("SBI misaligned access exception delegation ok\n"); + /* + * Note that we don't have to take any specific action here, if + * the delegation is successful, then + * check_unaligned_access_emulated() will verify that indeed the + * platform traps on misaligned accesses. + */ +} +#else +void __init unaligned_access_init(void) {} + +static int cpu_online_sbi_unaligned_setup(unsigned int cpu __always_unused) +{ + return 0; +} + #endif + +int cpu_online_unaligned_access_init(unsigned int cpu) +{ + int ret; + + ret = cpu_online_sbi_unaligned_setup(cpu); + if (ret) + return ret; + + return cpu_online_check_unaligned_access_emulated(cpu); +} + +bool misaligned_traps_can_delegate(void) +{ + /* + * Either we successfully requested misaligned traps delegation for all + * CPUs, or the SBI does not implement the FWFT extension but delegated + * the exception by default. + */ + return misaligned_traps_delegated || + all_cpus_unaligned_scalar_access_emulated(); +} +EXPORT_SYMBOL_GPL(misaligned_traps_can_delegate); diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index b8ba13819d05..ae2068425fbc 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -236,6 +236,11 @@ arch_initcall_sync(lock_and_set_unaligned_access_static_branch); static int riscv_online_cpu(unsigned int cpu) { + int ret = cpu_online_unaligned_access_init(cpu); + + if (ret) + return ret; + /* We are already set since the last check */ if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN) { goto exit; @@ -248,7 +253,6 @@ static int riscv_online_cpu(unsigned int cpu) { static struct page *buf; - check_unaligned_access_emulated(NULL); buf = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); if (!buf) { pr_warn("Allocation failure, not measuring misaligned performance\n"); @@ -439,6 +443,8 @@ static int __init check_unaligned_access_all_cpus(void) { int cpu; + unaligned_access_init(); + if (unaligned_scalar_speed_param != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN) { pr_info("scalar unaligned access speed set to '%s' (%lu) by command line\n", speed_str[unaligned_scalar_speed_param], unaligned_scalar_speed_param); diff --git a/arch/riscv/kernel/vdso.c b/arch/riscv/kernel/vdso.c index cc2895d1fbc2..3a8e038b10a2 100644 --- a/arch/riscv/kernel/vdso.c +++ b/arch/riscv/kernel/vdso.c @@ -136,7 +136,7 @@ static int __setup_additional_pages(struct mm_struct *mm, ret = _install_special_mapping(mm, vdso_base, vdso_text_len, - (VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC), + (VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC | VM_SEALED_SYSMAP), vdso_info->cm); if (IS_ERR(ret)) diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile index ad73607abc28..9ebb5e590f93 100644 --- a/arch/riscv/kernel/vdso/Makefile +++ b/arch/riscv/kernel/vdso/Makefile @@ -13,9 +13,17 @@ vdso-syms += flush_icache vdso-syms += hwprobe vdso-syms += sys_hwprobe +ifdef CONFIG_VDSO_GETRANDOM +vdso-syms += getrandom +endif + # Files to link into the vdso obj-vdso = $(patsubst %, %.o, $(vdso-syms)) note.o +ifdef CONFIG_VDSO_GETRANDOM +obj-vdso += vgetrandom-chacha.o +endif + ccflags-y := -fno-stack-protector ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -fno-builtin @@ -24,6 +32,10 @@ ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday.o += -fPIC -include $(c-gettimeofday-y) endif +ifneq ($(c-getrandom-y),) + CFLAGS_getrandom.o += -fPIC -include $(c-getrandom-y) +endif + CFLAGS_hwprobe.o += -fPIC # Build rules @@ -38,6 +50,7 @@ endif # Disable -pg to prevent insert call site CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) +CFLAGS_REMOVE_getrandom.o = $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) CFLAGS_REMOVE_hwprobe.o = $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) # Force dependency @@ -47,7 +60,7 @@ $(obj)/vdso.o: $(obj)/vdso.so $(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE $(call if_changed,vdsold_and_check) LDFLAGS_vdso.so.dbg = -shared -soname=linux-vdso.so.1 \ - --build-id=sha1 --hash-style=both --eh-frame-hdr + --build-id=sha1 --eh-frame-hdr # strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S diff --git a/arch/riscv/kernel/vdso/getrandom.c b/arch/riscv/kernel/vdso/getrandom.c new file mode 100644 index 000000000000..f21922e8cebd --- /dev/null +++ b/arch/riscv/kernel/vdso/getrandom.c @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2025 Xi Ruoyao <xry111@xry111.site>. All Rights Reserved. + */ +#include <linux/types.h> + +ssize_t __vdso_getrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state, size_t opaque_len) +{ + return __cvdso_getrandom(buffer, len, flags, opaque_state, opaque_len); +} diff --git a/arch/riscv/kernel/vdso/vdso.lds.S b/arch/riscv/kernel/vdso/vdso.lds.S index 8e86965a8aae..7c15b0f4ee3b 100644 --- a/arch/riscv/kernel/vdso/vdso.lds.S +++ b/arch/riscv/kernel/vdso/vdso.lds.S @@ -80,6 +80,9 @@ VERSION #ifndef COMPAT_VDSO __vdso_riscv_hwprobe; #endif +#if defined(CONFIG_VDSO_GETRANDOM) && !defined(COMPAT_VDSO) + __vdso_getrandom; +#endif local: *; }; } diff --git a/arch/riscv/kernel/vdso/vgetrandom-chacha.S b/arch/riscv/kernel/vdso/vgetrandom-chacha.S new file mode 100644 index 000000000000..5f0dad8f2373 --- /dev/null +++ b/arch/riscv/kernel/vdso/vgetrandom-chacha.S @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2025 Xi Ruoyao <xry111@xry111.site>. All Rights Reserved. + * + * Based on arch/loongarch/vdso/vgetrandom-chacha.S. + */ + +#include <asm/asm.h> +#include <linux/linkage.h> + +.text + +.macro ROTRI rd rs imm + slliw t0, \rs, 32 - \imm + srliw \rd, \rs, \imm + or \rd, \rd, t0 +.endm + +.macro OP_4REG op d0 d1 d2 d3 s0 s1 s2 s3 + \op \d0, \d0, \s0 + \op \d1, \d1, \s1 + \op \d2, \d2, \s2 + \op \d3, \d3, \s3 +.endm + +/* + * a0: output bytes + * a1: 32-byte key input + * a2: 8-byte counter input/output + * a3: number of 64-byte blocks to write to output + */ +SYM_FUNC_START(__arch_chacha20_blocks_nostack) + +#define output a0 +#define key a1 +#define counter a2 +#define nblocks a3 +#define i a4 +#define state0 s0 +#define state1 s1 +#define state2 s2 +#define state3 s3 +#define state4 s4 +#define state5 s5 +#define state6 s6 +#define state7 s7 +#define state8 s8 +#define state9 s9 +#define state10 s10 +#define state11 s11 +#define state12 a5 +#define state13 a6 +#define state14 a7 +#define state15 t1 +#define cnt t2 +#define copy0 t3 +#define copy1 t4 +#define copy2 t5 +#define copy3 t6 + +/* Packs to be used with OP_4REG */ +#define line0 state0, state1, state2, state3 +#define line1 state4, state5, state6, state7 +#define line2 state8, state9, state10, state11 +#define line3 state12, state13, state14, state15 + +#define line1_perm state5, state6, state7, state4 +#define line2_perm state10, state11, state8, state9 +#define line3_perm state15, state12, state13, state14 + +#define copy copy0, copy1, copy2, copy3 + +#define _16 16, 16, 16, 16 +#define _20 20, 20, 20, 20 +#define _24 24, 24, 24, 24 +#define _25 25, 25, 25, 25 + + /* + * The ABI requires s0-s9 saved. + * This does not violate the stack-less requirement: no sensitive data + * is spilled onto the stack. + */ + addi sp, sp, -12*SZREG + REG_S s0, (sp) + REG_S s1, SZREG(sp) + REG_S s2, 2*SZREG(sp) + REG_S s3, 3*SZREG(sp) + REG_S s4, 4*SZREG(sp) + REG_S s5, 5*SZREG(sp) + REG_S s6, 6*SZREG(sp) + REG_S s7, 7*SZREG(sp) + REG_S s8, 8*SZREG(sp) + REG_S s9, 9*SZREG(sp) + REG_S s10, 10*SZREG(sp) + REG_S s11, 11*SZREG(sp) + + ld cnt, (counter) + + li copy0, 0x61707865 + li copy1, 0x3320646e + li copy2, 0x79622d32 + li copy3, 0x6b206574 + +.Lblock: + /* state[0,1,2,3] = "expand 32-byte k" */ + mv state0, copy0 + mv state1, copy1 + mv state2, copy2 + mv state3, copy3 + + /* state[4,5,..,11] = key */ + lw state4, (key) + lw state5, 4(key) + lw state6, 8(key) + lw state7, 12(key) + lw state8, 16(key) + lw state9, 20(key) + lw state10, 24(key) + lw state11, 28(key) + + /* state[12,13] = counter */ + mv state12, cnt + srli state13, cnt, 32 + + /* state[14,15] = 0 */ + mv state14, zero + mv state15, zero + + li i, 10 +.Lpermute: + /* odd round */ + OP_4REG addw line0, line1 + OP_4REG xor line3, line0 + OP_4REG ROTRI line3, _16 + + OP_4REG addw line2, line3 + OP_4REG xor line1, line2 + OP_4REG ROTRI line1, _20 + + OP_4REG addw line0, line1 + OP_4REG xor line3, line0 + OP_4REG ROTRI line3, _24 + + OP_4REG addw line2, line3 + OP_4REG xor line1, line2 + OP_4REG ROTRI line1, _25 + + /* even round */ + OP_4REG addw line0, line1_perm + OP_4REG xor line3_perm, line0 + OP_4REG ROTRI line3_perm, _16 + + OP_4REG addw line2_perm, line3_perm + OP_4REG xor line1_perm, line2_perm + OP_4REG ROTRI line1_perm, _20 + + OP_4REG addw line0, line1_perm + OP_4REG xor line3_perm, line0 + OP_4REG ROTRI line3_perm, _24 + + OP_4REG addw line2_perm, line3_perm + OP_4REG xor line1_perm, line2_perm + OP_4REG ROTRI line1_perm, _25 + + addi i, i, -1 + bnez i, .Lpermute + + /* output[0,1,2,3] = copy[0,1,2,3] + state[0,1,2,3] */ + OP_4REG addw line0, copy + sw state0, (output) + sw state1, 4(output) + sw state2, 8(output) + sw state3, 12(output) + + /* from now on state[0,1,2,3] are scratch registers */ + + /* state[0,1,2,3] = lo(key) */ + lw state0, (key) + lw state1, 4(key) + lw state2, 8(key) + lw state3, 12(key) + + /* output[4,5,6,7] = state[0,1,2,3] + state[4,5,6,7] */ + OP_4REG addw line1, line0 + sw state4, 16(output) + sw state5, 20(output) + sw state6, 24(output) + sw state7, 28(output) + + /* state[0,1,2,3] = hi(key) */ + lw state0, 16(key) + lw state1, 20(key) + lw state2, 24(key) + lw state3, 28(key) + + /* output[8,9,10,11] = tmp[0,1,2,3] + state[8,9,10,11] */ + OP_4REG addw line2, line0 + sw state8, 32(output) + sw state9, 36(output) + sw state10, 40(output) + sw state11, 44(output) + + /* output[12,13,14,15] = state[12,13,14,15] + [cnt_lo, cnt_hi, 0, 0] */ + addw state12, state12, cnt + srli state0, cnt, 32 + addw state13, state13, state0 + sw state12, 48(output) + sw state13, 52(output) + sw state14, 56(output) + sw state15, 60(output) + + /* ++counter */ + addi cnt, cnt, 1 + + /* output += 64 */ + addi output, output, 64 + /* --nblocks */ + addi nblocks, nblocks, -1 + bnez nblocks, .Lblock + + /* counter = [cnt_lo, cnt_hi] */ + sd cnt, (counter) + + /* Zero out the potentially sensitive regs, in case nothing uses these + * again. As at now copy[0,1,2,3] just contains "expand 32-byte k" and + * state[0,...,11] are s0-s11 those we'll restore in the epilogue, we + * only need to zero state[12,...,15]. + */ + mv state12, zero + mv state13, zero + mv state14, zero + mv state15, zero + + REG_L s0, (sp) + REG_L s1, SZREG(sp) + REG_L s2, 2*SZREG(sp) + REG_L s3, 3*SZREG(sp) + REG_L s4, 4*SZREG(sp) + REG_L s5, 5*SZREG(sp) + REG_L s6, 6*SZREG(sp) + REG_L s7, 7*SZREG(sp) + REG_L s8, 8*SZREG(sp) + REG_L s9, 9*SZREG(sp) + REG_L s10, 10*SZREG(sp) + REG_L s11, 11*SZREG(sp) + addi sp, sp, 12*SZREG + + ret +SYM_FUNC_END(__arch_chacha20_blocks_nostack) diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c index 9feb7f67a0a3..92d8ff81f42c 100644 --- a/arch/riscv/kernel/vendor_extensions.c +++ b/arch/riscv/kernel/vendor_extensions.c @@ -6,6 +6,7 @@ #include <asm/vendorid_list.h> #include <asm/vendor_extensions.h> #include <asm/vendor_extensions/andes.h> +#include <asm/vendor_extensions/sifive.h> #include <asm/vendor_extensions/thead.h> #include <linux/array_size.h> @@ -15,6 +16,9 @@ struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_ANDES &riscv_isa_vendor_ext_list_andes, #endif +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_SIFIVE + &riscv_isa_vendor_ext_list_sifive, +#endif #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD &riscv_isa_vendor_ext_list_thead, #endif @@ -45,6 +49,12 @@ bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsig cpu_bmap = riscv_isa_vendor_ext_list_andes.per_hart_isa_bitmap; break; #endif + #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_SIFIVE + case SIFIVE_VENDOR_ID: + bmap = &riscv_isa_vendor_ext_list_sifive.all_harts_isa_bitmap; + cpu_bmap = riscv_isa_vendor_ext_list_sifive.per_hart_isa_bitmap; + break; + #endif #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD case THEAD_VENDOR_ID: bmap = &riscv_isa_vendor_ext_list_thead.all_harts_isa_bitmap; diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile index 866414c81a9f..a4eca96d1c8a 100644 --- a/arch/riscv/kernel/vendor_extensions/Makefile +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -1,5 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_ANDES) += andes.o +obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_SIFIVE) += sifive.o +obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_SIFIVE) += sifive_hwprobe.o obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead_hwprobe.o diff --git a/arch/riscv/kernel/vendor_extensions/sifive.c b/arch/riscv/kernel/vendor_extensions/sifive.c new file mode 100644 index 000000000000..1411337dc1e6 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/sifive.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <asm/cpufeature.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/sifive.h> + +#include <linux/array_size.h> +#include <linux/types.h> + +/* All SiFive vendor extensions supported in Linux */ +const struct riscv_isa_ext_data riscv_isa_vendor_ext_sifive[] = { + __RISCV_ISA_EXT_DATA(xsfvfnrclipxfqf, RISCV_ISA_VENDOR_EXT_XSFVFNRCLIPXFQF), + __RISCV_ISA_EXT_DATA(xsfvfwmaccqqq, RISCV_ISA_VENDOR_EXT_XSFVFWMACCQQQ), + __RISCV_ISA_EXT_DATA(xsfvqmaccdod, RISCV_ISA_VENDOR_EXT_XSFVQMACCDOD), + __RISCV_ISA_EXT_DATA(xsfvqmaccqoq, RISCV_ISA_VENDOR_EXT_XSFVQMACCQOQ), +}; + +struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_sifive = { + .ext_data_count = ARRAY_SIZE(riscv_isa_vendor_ext_sifive), + .ext_data = riscv_isa_vendor_ext_sifive, +}; diff --git a/arch/riscv/kernel/vendor_extensions/sifive_hwprobe.c b/arch/riscv/kernel/vendor_extensions/sifive_hwprobe.c new file mode 100644 index 000000000000..1f77f6309763 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/sifive_hwprobe.c @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <asm/vendor_extensions/sifive.h> +#include <asm/vendor_extensions/sifive_hwprobe.h> +#include <asm/vendor_extensions/vendor_hwprobe.h> + +#include <linux/cpumask.h> +#include <linux/types.h> + +#include <uapi/asm/hwprobe.h> +#include <uapi/asm/vendor/sifive.h> + +void hwprobe_isa_vendor_ext_sifive_0(struct riscv_hwprobe *pair, const struct cpumask *cpus) +{ + VENDOR_EXTENSION_SUPPORTED(pair, cpus, + riscv_isa_vendor_ext_list_sifive.per_hart_isa_bitmap, { + VENDOR_EXT_KEY(XSFVQMACCDOD); + VENDOR_EXT_KEY(XSFVQMACCQOQ); + VENDOR_EXT_KEY(XSFVFNRCLIPXFQF); + VENDOR_EXT_KEY(XSFVFWMACCQQQ); + }); +} diff --git a/arch/riscv/lib/riscv_v_helpers.c b/arch/riscv/lib/riscv_v_helpers.c index be38a93cedae..7bbdfc6d4552 100644 --- a/arch/riscv/lib/riscv_v_helpers.c +++ b/arch/riscv/lib/riscv_v_helpers.c @@ -16,8 +16,11 @@ #ifdef CONFIG_MMU size_t riscv_v_usercopy_threshold = CONFIG_RISCV_ISA_V_UCOPY_THRESHOLD; int __asm_vector_usercopy(void *dst, void *src, size_t n); +int __asm_vector_usercopy_sum_enabled(void *dst, void *src, size_t n); int fallback_scalar_usercopy(void *dst, void *src, size_t n); -asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n) +int fallback_scalar_usercopy_sum_enabled(void *dst, void *src, size_t n); +asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n, + bool enable_sum) { size_t remain, copied; @@ -26,7 +29,8 @@ asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n) goto fallback; kernel_vector_begin(); - remain = __asm_vector_usercopy(dst, src, n); + remain = enable_sum ? __asm_vector_usercopy(dst, src, n) : + __asm_vector_usercopy_sum_enabled(dst, src, n); kernel_vector_end(); if (remain) { @@ -40,6 +44,7 @@ asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n) return remain; fallback: - return fallback_scalar_usercopy(dst, src, n); + return enable_sum ? fallback_scalar_usercopy(dst, src, n) : + fallback_scalar_usercopy_sum_enabled(dst, src, n); } #endif diff --git a/arch/riscv/lib/uaccess.S b/arch/riscv/lib/uaccess.S index 6a9f116bb545..4efea1b3326c 100644 --- a/arch/riscv/lib/uaccess.S +++ b/arch/riscv/lib/uaccess.S @@ -17,14 +17,43 @@ SYM_FUNC_START(__asm_copy_to_user) ALTERNATIVE("j fallback_scalar_usercopy", "nop", 0, RISCV_ISA_EXT_ZVE32X, CONFIG_RISCV_ISA_V) REG_L t0, riscv_v_usercopy_threshold bltu a2, t0, fallback_scalar_usercopy - tail enter_vector_usercopy + li a3, 1 + tail enter_vector_usercopy #endif -SYM_FUNC_START(fallback_scalar_usercopy) +SYM_FUNC_END(__asm_copy_to_user) +EXPORT_SYMBOL(__asm_copy_to_user) +SYM_FUNC_ALIAS(__asm_copy_from_user, __asm_copy_to_user) +EXPORT_SYMBOL(__asm_copy_from_user) +SYM_FUNC_START(fallback_scalar_usercopy) /* Enable access to user memory */ - li t6, SR_SUM - csrs CSR_STATUS, t6 + li t6, SR_SUM + csrs CSR_STATUS, t6 + mv t6, ra + call fallback_scalar_usercopy_sum_enabled + + /* Disable access to user memory */ + mv ra, t6 + li t6, SR_SUM + csrc CSR_STATUS, t6 + ret +SYM_FUNC_END(fallback_scalar_usercopy) + +SYM_FUNC_START(__asm_copy_to_user_sum_enabled) +#ifdef CONFIG_RISCV_ISA_V + ALTERNATIVE("j fallback_scalar_usercopy_sum_enabled", "nop", 0, RISCV_ISA_EXT_ZVE32X, CONFIG_RISCV_ISA_V) + REG_L t0, riscv_v_usercopy_threshold + bltu a2, t0, fallback_scalar_usercopy_sum_enabled + li a3, 0 + tail enter_vector_usercopy +#endif +SYM_FUNC_END(__asm_copy_to_user_sum_enabled) +SYM_FUNC_ALIAS(__asm_copy_from_user_sum_enabled, __asm_copy_to_user_sum_enabled) +EXPORT_SYMBOL(__asm_copy_from_user_sum_enabled) +EXPORT_SYMBOL(__asm_copy_to_user_sum_enabled) + +SYM_FUNC_START(fallback_scalar_usercopy_sum_enabled) /* * Save the terminal address which will be used to compute the number * of bytes copied in case of a fixup exception. @@ -178,23 +207,12 @@ SYM_FUNC_START(fallback_scalar_usercopy) bltu a0, t0, 4b /* t0 - end of dst */ .Lout_copy_user: - /* Disable access to user memory */ - csrc CSR_STATUS, t6 li a0, 0 ret - - /* Exception fixup code */ 10: - /* Disable access to user memory */ - csrc CSR_STATUS, t6 sub a0, t5, a0 ret -SYM_FUNC_END(__asm_copy_to_user) -SYM_FUNC_END(fallback_scalar_usercopy) -EXPORT_SYMBOL(__asm_copy_to_user) -SYM_FUNC_ALIAS(__asm_copy_from_user, __asm_copy_to_user) -EXPORT_SYMBOL(__asm_copy_from_user) - +SYM_FUNC_END(fallback_scalar_usercopy_sum_enabled) SYM_FUNC_START(__clear_user) diff --git a/arch/riscv/lib/uaccess_vector.S b/arch/riscv/lib/uaccess_vector.S index 7c45f26de4f7..03b5560609a2 100644 --- a/arch/riscv/lib/uaccess_vector.S +++ b/arch/riscv/lib/uaccess_vector.S @@ -24,7 +24,18 @@ SYM_FUNC_START(__asm_vector_usercopy) /* Enable access to user memory */ li t6, SR_SUM csrs CSR_STATUS, t6 + mv t6, ra + call __asm_vector_usercopy_sum_enabled + + /* Disable access to user memory */ + mv ra, t6 + li t6, SR_SUM + csrc CSR_STATUS, t6 + ret +SYM_FUNC_END(__asm_vector_usercopy) + +SYM_FUNC_START(__asm_vector_usercopy_sum_enabled) loop: vsetvli iVL, iNum, e8, ELEM_LMUL_SETTING, ta, ma fixup vle8.v vData, (pSrc), 10f @@ -36,8 +47,6 @@ loop: /* Exception fixup for vector load is shared with normal exit */ 10: - /* Disable access to user memory */ - csrc CSR_STATUS, t6 mv a0, iNum ret @@ -49,4 +58,4 @@ loop: csrr t2, CSR_VSTART sub iNum, iNum, t2 j 10b -SYM_FUNC_END(__asm_vector_usercopy) +SYM_FUNC_END(__asm_vector_usercopy_sum_enabled) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index b8e96dfff19d..4ca5aafce22e 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -24,7 +24,20 @@ void flush_icache_all(void) if (num_online_cpus() < 2) return; - else if (riscv_use_sbi_for_rfence()) + + /* + * Make sure all previous writes to the D$ are ordered before making + * the IPI. The RISC-V spec states that a hart must execute a data fence + * before triggering a remote fence.i in order to make the modification + * visable for remote harts. + * + * IPIs on RISC-V are triggered by MMIO writes to either CLINT or + * S-IMSIC, so the fence ensures previous data writes "happen before" + * the MMIO. + */ + RISCV_FENCE(w, o); + + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); @@ -101,6 +114,9 @@ EXPORT_SYMBOL_GPL(riscv_cbom_block_size); unsigned int riscv_cboz_block_size; EXPORT_SYMBOL_GPL(riscv_cboz_block_size); +unsigned int riscv_cbop_block_size; +EXPORT_SYMBOL_GPL(riscv_cbop_block_size); + static void __init cbo_get_block_size(struct device_node *node, const char *name, u32 *block_size, unsigned long *first_hartid) @@ -125,8 +141,8 @@ static void __init cbo_get_block_size(struct device_node *node, void __init riscv_init_cbo_blocksizes(void) { - unsigned long cbom_hartid, cboz_hartid; - u32 cbom_block_size = 0, cboz_block_size = 0; + unsigned long cbom_hartid, cboz_hartid, cbop_hartid; + u32 cbom_block_size = 0, cboz_block_size = 0, cbop_block_size = 0; struct device_node *node; struct acpi_table_header *rhct; acpi_status status; @@ -138,13 +154,15 @@ void __init riscv_init_cbo_blocksizes(void) &cbom_block_size, &cbom_hartid); cbo_get_block_size(node, "riscv,cboz-block-size", &cboz_block_size, &cboz_hartid); + cbo_get_block_size(node, "riscv,cbop-block-size", + &cbop_block_size, &cbop_hartid); } } else { status = acpi_get_table(ACPI_SIG_RHCT, 0, &rhct); if (ACPI_FAILURE(status)) return; - acpi_get_cbo_block_size(rhct, &cbom_block_size, &cboz_block_size, NULL); + acpi_get_cbo_block_size(rhct, &cbom_block_size, &cboz_block_size, &cbop_block_size); acpi_put_table((struct acpi_table_header *)rhct); } @@ -153,6 +171,9 @@ void __init riscv_init_cbo_blocksizes(void) if (cboz_block_size) riscv_cboz_block_size = cboz_block_size; + + if (cbop_block_size) + riscv_cbop_block_size = cbop_block_size; } #ifdef CONFIG_SMP diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 4ae67324f992..8b6c0a112a8d 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -154,4 +154,14 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, flush_tlb_mm(vma->vm_mm); return pmd; } + +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + VM_WARN_ON_ONCE(!pud_present(*pudp)); + pud_t old = pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp)); + + flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE); + return old; +} #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index f9e27ba1df99..e737ba7949b1 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,6 +7,27 @@ #include <linux/mmu_notifier.h> #include <asm/sbi.h> #include <asm/mmu_context.h> +#include <asm/cpufeature.h> + +#define has_svinval() riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL) + +static inline void local_sfence_inval_ir(void) +{ + asm volatile(SFENCE_INVAL_IR() ::: "memory"); +} + +static inline void local_sfence_w_inval(void) +{ + asm volatile(SFENCE_W_INVAL() ::: "memory"); +} + +static inline void local_sinval_vma(unsigned long vma, unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + asm volatile(SINVAL_VMA(%0, %1) : : "r" (vma), "r" (asid) : "memory"); + else + asm volatile(SINVAL_VMA(%0, zero) : : "r" (vma) : "memory"); +} /* * Flush entire TLB if number of entries to be flushed is greater @@ -27,6 +48,16 @@ static void local_flush_tlb_range_threshold_asid(unsigned long start, return; } + if (has_svinval()) { + local_sfence_w_inval(); + for (i = 0; i < nr_ptes_in_range; ++i) { + local_sinval_vma(start, asid); + start += stride; + } + local_sfence_inval_ir(); + return; + } + for (i = 0; i < nr_ptes_in_range; ++i) { local_flush_tlb_page_asid(start, asid); start += stride; @@ -182,6 +213,13 @@ void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, __flush_tlb_range(vma->vm_mm, mm_cpumask(vma->vm_mm), start, end - start, PMD_SIZE); } + +void flush_pud_tlb_range(struct vm_area_struct *vma, unsigned long start, + unsigned long end) +{ + __flush_tlb_range(vma->vm_mm, mm_cpumask(vma->vm_mm), + start, end - start, PUD_SIZE); +} #endif bool arch_tlbbatch_should_defer(struct mm_struct *mm) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index 1c661ac62ce8..6d8bc27a366e 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -915,7 +915,7 @@ static inline int pmd_protnone(pmd_t pmd) } #endif -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile index db5f3a3faefb..ea5ed6654050 100644 --- a/arch/s390/kernel/Makefile +++ b/arch/s390/kernel/Makefile @@ -46,7 +46,7 @@ obj-y += nospec-branch.o ipl_vmparm.o machine_kexec_reloc.o unwind_bc.o obj-y += smp.o text_amode31.o stacktrace.o abs_lowcore.o facility.o uv.o wti.o obj-y += diag/ -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds obj-$(CONFIG_SYSFS) += nospec-sysfs.o CFLAGS_REMOVE_nospec-branch.o += $(CC_FLAGS_EXPOLINE) diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c index e23670e1949c..21c2e61fece4 100644 --- a/arch/s390/kvm/gaccess.c +++ b/arch/s390/kvm/gaccess.c @@ -319,7 +319,7 @@ enum prot_type { PROT_TYPE_DAT = 3, PROT_TYPE_IEP = 4, /* Dummy value for passing an initialized value when code != PGM_PROTECTION */ - PROT_NONE, + PROT_TYPE_DUMMY, }; static int trans_exc_ending(struct kvm_vcpu *vcpu, int code, unsigned long gva, u8 ar, @@ -335,7 +335,7 @@ static int trans_exc_ending(struct kvm_vcpu *vcpu, int code, unsigned long gva, switch (code) { case PGM_PROTECTION: switch (prot) { - case PROT_NONE: + case PROT_TYPE_DUMMY: /* We should never get here, acts like termination */ WARN_ON_ONCE(1); break; @@ -805,7 +805,7 @@ static int guest_range_to_gpas(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, gpa = kvm_s390_real_to_abs(vcpu, ga); if (!kvm_is_gpa_in_memslot(vcpu->kvm, gpa)) { rc = PGM_ADDRESSING; - prot = PROT_NONE; + prot = PROT_TYPE_DUMMY; } } if (rc) @@ -963,7 +963,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, if (rc == PGM_PROTECTION) prot = PROT_TYPE_KEYC; else - prot = PROT_NONE; + prot = PROT_TYPE_DUMMY; rc = trans_exc_ending(vcpu, rc, ga, ar, mode, prot, terminate); } out_unlock: diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 3829521450dd..e1ad05bfd28a 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -441,6 +441,8 @@ void do_secure_storage_access(struct pt_regs *regs) if (rc) BUG(); } else { + if (faulthandler_disabled()) + return handle_fault_error_nolock(regs, 0); mm = current->mm; mmap_read_lock(mm); vma = find_vma(mm, addr); diff --git a/arch/sh/boards/mach-ecovec24/setup.c b/arch/sh/boards/mach-ecovec24/setup.c index 6f13557eecd6..a641e26f8fdf 100644 --- a/arch/sh/boards/mach-ecovec24/setup.c +++ b/arch/sh/boards/mach-ecovec24/setup.c @@ -825,6 +825,7 @@ static struct spi_board_info spi_bus[] = { /* MSIOF0 */ static struct sh_msiof_spi_info msiof0_data = { .num_chipselect = 1, + .mode = MSIOF_SPI_HOST, }; static struct resource msiof0_resources[] = { diff --git a/arch/sh/drivers/heartbeat.c b/arch/sh/drivers/heartbeat.c index 24391b444b28..42103038a7d0 100644 --- a/arch/sh/drivers/heartbeat.c +++ b/arch/sh/drivers/heartbeat.c @@ -58,7 +58,7 @@ static inline void heartbeat_toggle_bit(struct heartbeat_data *hd, static void heartbeat_timer(struct timer_list *t) { - struct heartbeat_data *hd = from_timer(hd, t, timer); + struct heartbeat_data *hd = timer_container_of(hd, t, timer); static unsigned bit = 0, up = 1; heartbeat_toggle_bit(hd, bit, hd->flags & HEARTBEAT_INVERTED); diff --git a/arch/sh/drivers/pci/common.c b/arch/sh/drivers/pci/common.c index 5442475d132e..9633b6147a05 100644 --- a/arch/sh/drivers/pci/common.c +++ b/arch/sh/drivers/pci/common.c @@ -88,7 +88,7 @@ int __init pci_is_66mhz_capable(struct pci_channel *hose, static void pcibios_enable_err(struct timer_list *t) { - struct pci_channel *hose = from_timer(hose, t, err_timer); + struct pci_channel *hose = timer_container_of(hose, t, err_timer); timer_delete(&hose->err_timer); printk(KERN_DEBUG "PCI: re-enabling error IRQ.\n"); @@ -97,7 +97,7 @@ static void pcibios_enable_err(struct timer_list *t) static void pcibios_enable_serr(struct timer_list *t) { - struct pci_channel *hose = from_timer(hose, t, serr_timer); + struct pci_channel *hose = timer_container_of(hose, t, serr_timer); timer_delete(&hose->serr_timer); printk(KERN_DEBUG "PCI: re-enabling system error IRQ.\n"); diff --git a/arch/sh/drivers/push-switch.c b/arch/sh/drivers/push-switch.c index 2b51ad9d5586..443cc6fd26a8 100644 --- a/arch/sh/drivers/push-switch.c +++ b/arch/sh/drivers/push-switch.c @@ -25,7 +25,7 @@ static DEVICE_ATTR_RO(switch); static void switch_timer(struct timer_list *t) { - struct push_switch *psw = from_timer(psw, t, debounce); + struct push_switch *psw = timer_container_of(psw, t, debounce); schedule_work(&psw->work); } diff --git a/arch/sh/include/asm/cache.h b/arch/sh/include/asm/cache.h index b38dbc975581..e7ac9c950275 100644 --- a/arch/sh/include/asm/cache.h +++ b/arch/sh/include/asm/cache.h @@ -22,7 +22,7 @@ #define __read_mostly __section(".data..read_mostly") -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ struct cache_info { unsigned int ways; /* Number of cache ways */ unsigned int sets; /* Number of cache sets */ @@ -48,5 +48,5 @@ struct cache_info { unsigned long flags; }; -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_CACHE_H */ diff --git a/arch/sh/include/asm/dwarf.h b/arch/sh/include/asm/dwarf.h index 571954474122..f46d18b84833 100644 --- a/arch/sh/include/asm/dwarf.h +++ b/arch/sh/include/asm/dwarf.h @@ -189,7 +189,7 @@ */ #define DWARF_ARCH_RA_REG 17 -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/compiler.h> #include <linux/bug.h> @@ -379,7 +379,7 @@ extern int module_dwarf_finalize(const Elf_Ehdr *, const Elf_Shdr *, struct module *); extern void module_dwarf_cleanup(struct module *); -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #define CFI_STARTPROC .cfi_startproc #define CFI_ENDPROC .cfi_endproc @@ -402,7 +402,7 @@ extern void module_dwarf_cleanup(struct module *); #define CFI_REL_OFFSET CFI_IGNORE #define CFI_UNDEFINED CFI_IGNORE -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ static inline void dwarf_unwinder_init(void) { } diff --git a/arch/sh/include/asm/fpu.h b/arch/sh/include/asm/fpu.h index 0379f4cce5ed..a086e38b70ee 100644 --- a/arch/sh/include/asm/fpu.h +++ b/arch/sh/include/asm/fpu.h @@ -2,7 +2,7 @@ #ifndef __ASM_SH_FPU_H #define __ASM_SH_FPU_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/ptrace.h> @@ -67,6 +67,6 @@ static inline void clear_fpu(struct task_struct *tsk, struct pt_regs *regs) void float_raise(unsigned int flags); int float_rounding_mode(void); -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_FPU_H */ diff --git a/arch/sh/include/asm/ftrace.h b/arch/sh/include/asm/ftrace.h index 1c10e1066390..d35781ab716e 100644 --- a/arch/sh/include/asm/ftrace.h +++ b/arch/sh/include/asm/ftrace.h @@ -7,7 +7,7 @@ #define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */ #define FTRACE_SYSCALL_MAX NR_syscalls -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ extern void mcount(void); #define MCOUNT_ADDR ((unsigned long)(mcount)) @@ -35,10 +35,10 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr) void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr); -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* CONFIG_FUNCTION_TRACER */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* arch/sh/kernel/return_address.c */ extern void *return_address(unsigned int); @@ -53,6 +53,6 @@ static inline void arch_ftrace_nmi_enter(void) { } static inline void arch_ftrace_nmi_exit(void) { } #endif -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_FTRACE_H */ diff --git a/arch/sh/include/asm/mmu.h b/arch/sh/include/asm/mmu.h index 172e329fd92d..b9c9f91e6616 100644 --- a/arch/sh/include/asm/mmu.h +++ b/arch/sh/include/asm/mmu.h @@ -33,7 +33,7 @@ #define PMB_NO_ENTRY (-1) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/errno.h> #include <linux/threads.h> #include <asm/page.h> @@ -102,6 +102,6 @@ pmb_remap(phys_addr_t phys, unsigned long size, pgprot_t prot) return pmb_remap_caller(phys, size, prot, __builtin_return_address(0)); } -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __MMU_H */ diff --git a/arch/sh/include/asm/page.h b/arch/sh/include/asm/page.h index 3990cbd9aa04..def4205491ec 100644 --- a/arch/sh/include/asm/page.h +++ b/arch/sh/include/asm/page.h @@ -30,7 +30,7 @@ #define HUGETLB_PAGE_ORDER (HPAGE_SHIFT-PAGE_SHIFT) #endif -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/uncached.h> extern unsigned long shm_align_mask; @@ -85,7 +85,7 @@ typedef struct page *pgtable_t; #define pte_pgprot(x) __pgprot(pte_val(x) & PTE_FLAGS_MASK) -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ /* * __MEMORY_START and SIZE are the physical addresses and size of RAM. @@ -126,10 +126,10 @@ typedef struct page *pgtable_t; #define ___va(x) ((x)+PAGE_OFFSET) #endif -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #define __pa(x) ___pa((unsigned long)x) #define __va(x) (void *)___va((unsigned long)x) -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #ifdef CONFIG_UNCACHED_MAPPING #if defined(CONFIG_29BIT) diff --git a/arch/sh/include/asm/pgtable.h b/arch/sh/include/asm/pgtable.h index 729f5c6225fb..10fa8f2bb8d1 100644 --- a/arch/sh/include/asm/pgtable.h +++ b/arch/sh/include/asm/pgtable.h @@ -17,7 +17,7 @@ #include <asm/page.h> #include <asm/mmu.h> -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/addrspace.h> #include <asm/fixmap.h> @@ -28,7 +28,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)]; #define ZERO_PAGE(vaddr) (virt_to_page(empty_zero_page)) -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ /* * Effective and physical address definitions, to aid with sign diff --git a/arch/sh/include/asm/pgtable_32.h b/arch/sh/include/asm/pgtable_32.h index db2e48366e0d..5f51af18997b 100644 --- a/arch/sh/include/asm/pgtable_32.h +++ b/arch/sh/include/asm/pgtable_32.h @@ -170,7 +170,7 @@ static inline unsigned long copy_ptea_attributes(unsigned long x) (PTE_MASK | _PAGE_ACCESSED | _PAGE_CACHABLE | \ _PAGE_DIRTY | _PAGE_SPECIAL) -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #if defined(CONFIG_X2TLB) /* SH-X2 TLB */ #define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_CACHABLE | \ @@ -287,9 +287,9 @@ static inline unsigned long copy_ptea_attributes(unsigned long x) __pgprot(0) #endif -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* * Certain architectures need to do special things when PTEs @@ -470,7 +470,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) /* In both cases, we borrow bit 6 to store the exclusive marker in swap PTEs. */ #define _PAGE_SWP_EXCLUSIVE _PAGE_USER -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte.pte_low & _PAGE_SWP_EXCLUSIVE; } @@ -478,5 +478,5 @@ static inline int pte_swp_exclusive(pte_t pte) PTE_BIT_FUNC(low, swp_mkexclusive, |= _PAGE_SWP_EXCLUSIVE); PTE_BIT_FUNC(low, swp_clear_exclusive, &= ~_PAGE_SWP_EXCLUSIVE); -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_PGTABLE_32_H */ diff --git a/arch/sh/include/asm/processor.h b/arch/sh/include/asm/processor.h index 73fba7c922f9..2a0b5713ab80 100644 --- a/arch/sh/include/asm/processor.h +++ b/arch/sh/include/asm/processor.h @@ -5,7 +5,7 @@ #include <asm/cpu-features.h> #include <asm/cache.h> -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ /* * CPU type and hardware bug flags. Kept separately for each CPU. * @@ -168,7 +168,7 @@ extern unsigned int instruction_size(unsigned int insn); void select_idle_routine(void); -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #include <asm/processor_32.h> diff --git a/arch/sh/include/asm/smc37c93x.h b/arch/sh/include/asm/smc37c93x.h index 891f2f8f2fd0..caf4cd8dd241 100644 --- a/arch/sh/include/asm/smc37c93x.h +++ b/arch/sh/include/asm/smc37c93x.h @@ -67,7 +67,7 @@ #define UART_DLL 0x0 /* Divisor Latch (LS) */ #define UART_DLM 0x2 /* Divisor Latch (MS) */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ typedef struct uart_reg { volatile __u16 rbr; volatile __u16 ier; @@ -78,7 +78,7 @@ typedef struct uart_reg { volatile __u16 msr; volatile __u16 scr; } uart_reg; -#endif /* ! __ASSEMBLY__ */ +#endif /* ! __ASSEMBLER__ */ /* Alias for Write Only Register */ diff --git a/arch/sh/include/asm/suspend.h b/arch/sh/include/asm/suspend.h index 47db17520261..0f991babc559 100644 --- a/arch/sh/include/asm/suspend.h +++ b/arch/sh/include/asm/suspend.h @@ -2,7 +2,7 @@ #ifndef _ASM_SH_SUSPEND_H #define _ASM_SH_SUSPEND_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/notifier.h> #include <asm/ptrace.h> diff --git a/arch/sh/include/asm/thread_info.h b/arch/sh/include/asm/thread_info.h index 9f19a682d315..471db5173036 100644 --- a/arch/sh/include/asm/thread_info.h +++ b/arch/sh/include/asm/thread_info.h @@ -21,7 +21,7 @@ #define FAULT_CODE_PROT (1 << 3) /* protection fault */ #define FAULT_CODE_USER (1 << 4) /* user-mode access */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <asm/processor.h> struct thread_info { @@ -49,7 +49,7 @@ struct thread_info { /* * macros/functions for gaining access to the thread information structure */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #define INIT_THREAD_INFO(tsk) \ { \ .task = &tsk, \ @@ -86,7 +86,7 @@ static inline struct thread_info *current_thread_info(void) extern void init_thread_xstate(void); -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ /* * Thread information flags @@ -144,7 +144,7 @@ extern void init_thread_xstate(void); */ #define TS_USEDFPU 0x0002 /* FPU used by this task this quantum */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #define TI_FLAG_FAULT_CODE_SHIFT 24 @@ -164,5 +164,5 @@ static inline unsigned int get_thread_fault_code(void) return ti->flags >> TI_FLAG_FAULT_CODE_SHIFT; } -#endif /* !__ASSEMBLY__ */ +#endif /* !__ASSEMBLER__ */ #endif /* __ASM_SH_THREAD_INFO_H */ diff --git a/arch/sh/include/asm/tlb.h b/arch/sh/include/asm/tlb.h index ddf324bfb9a0..39df40d0ebc2 100644 --- a/arch/sh/include/asm/tlb.h +++ b/arch/sh/include/asm/tlb.h @@ -2,7 +2,7 @@ #ifndef __ASM_SH_TLB_H #define __ASM_SH_TLB_H -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ #include <linux/pagemap.h> #include <asm-generic/tlb.h> @@ -29,5 +29,5 @@ asmlinkage int handle_tlbmiss(struct pt_regs *regs, unsigned long error_code, unsigned long address); #endif /* CONFIG_MMU */ -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_TLB_H */ diff --git a/arch/sh/include/asm/types.h b/arch/sh/include/asm/types.h index 9b3fc923ee28..fec3e89df0b1 100644 --- a/arch/sh/include/asm/types.h +++ b/arch/sh/include/asm/types.h @@ -7,10 +7,10 @@ /* * These aren't exported outside the kernel to avoid name space clashes */ -#ifndef __ASSEMBLY__ +#ifndef __ASSEMBLER__ typedef u16 insn_size_t; typedef u32 reg_size_t; -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ #endif /* __ASM_SH_TYPES_H */ diff --git a/arch/sh/include/mach-common/mach/romimage.h b/arch/sh/include/mach-common/mach/romimage.h index 1915714263aa..22fb47ec9b15 100644 --- a/arch/sh/include/mach-common/mach/romimage.h +++ b/arch/sh/include/mach-common/mach/romimage.h @@ -1,12 +1,12 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ /* do nothing here by default */ -#else /* __ASSEMBLY__ */ +#else /* __ASSEMBLER__ */ static inline void mmcif_update_progress(int nr) { } -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ diff --git a/arch/sh/include/mach-ecovec24/mach/romimage.h b/arch/sh/include/mach-ecovec24/mach/romimage.h index 2da6ff326cbd..f93d494736c3 100644 --- a/arch/sh/include/mach-ecovec24/mach/romimage.h +++ b/arch/sh/include/mach-ecovec24/mach/romimage.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ /* EcoVec board specific boot code: * converts the "partner-jet-script.txt" script into assembly @@ -22,7 +22,7 @@ 1 : .long 0xa8000000 2 : -#else /* __ASSEMBLY__ */ +#else /* __ASSEMBLER__ */ /* Ecovec board specific information: * @@ -45,4 +45,4 @@ static inline void mmcif_update_progress(int nr) __raw_writeb(1 << (nr - 1), PGDR); } -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ diff --git a/arch/sh/include/mach-kfr2r09/mach/romimage.h b/arch/sh/include/mach-kfr2r09/mach/romimage.h index 209275872ff0..f68bb480d378 100644 --- a/arch/sh/include/mach-kfr2r09/mach/romimage.h +++ b/arch/sh/include/mach-kfr2r09/mach/romimage.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifdef __ASSEMBLY__ +#ifdef __ASSEMBLER__ /* kfr2r09 board specific boot code: * converts the "partner-jet-script.txt" script into assembly @@ -22,10 +22,10 @@ 1: .long 0xa8000000 2: -#else /* __ASSEMBLY__ */ +#else /* __ASSEMBLER__ */ static inline void mmcif_update_progress(int nr) { } -#endif /* __ASSEMBLY__ */ +#endif /* __ASSEMBLER__ */ diff --git a/arch/sh/kernel/Makefile b/arch/sh/kernel/Makefile index 7b453592adaf..5ef123bc63f8 100644 --- a/arch/sh/kernel/Makefile +++ b/arch/sh/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the Linux/SuperH kernel. # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds ifdef CONFIG_FUNCTION_TRACER # Do not profile debug and lowlevel utilities diff --git a/arch/sh/kernel/kprobes.c b/arch/sh/kernel/kprobes.c index 49c4ffd782d6..a250fb1b9420 100644 --- a/arch/sh/kernel/kprobes.c +++ b/arch/sh/kernel/kprobes.c @@ -404,13 +404,10 @@ int __kprobes kprobe_fault_handler(struct pt_regs *regs, int trapnr) int __kprobes kprobe_exceptions_notify(struct notifier_block *self, unsigned long val, void *data) { - struct kprobe *p = NULL; struct die_args *args = (struct die_args *)data; int ret = NOTIFY_DONE; - kprobe_opcode_t *addr = NULL; struct kprobe_ctlblk *kcb = get_kprobe_ctlblk(); - addr = (kprobe_opcode_t *) (args->regs->pc); if (val == DIE_TRAP && args->trapnr == (BREAKPOINT_INSTRUCTION & 0xff)) { if (!kprobe_running()) { @@ -421,7 +418,6 @@ int __kprobes kprobe_exceptions_notify(struct notifier_block *self, ret = NOTIFY_DONE; } } else { - p = get_kprobe(addr); if ((kcb->kprobe_status == KPROBE_HIT_SS) || (kcb->kprobe_status == KPROBE_REENTER)) { if (post_kprobe_handler(args->regs)) diff --git a/arch/sparc/include/asm/pgtable_32.h b/arch/sparc/include/asm/pgtable_32.h index 1454ebe91539..7c199c003ffe 100644 --- a/arch/sparc/include/asm/pgtable_32.h +++ b/arch/sparc/include/asm/pgtable_32.h @@ -348,7 +348,7 @@ static inline swp_entry_t __swp_entry(unsigned long type, unsigned long offset) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & SRMMU_SWP_EXCLUSIVE; } diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index 4af03e3c161b..669cd02469a1 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -1023,7 +1023,7 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/sparc/kernel/Makefile b/arch/sparc/kernel/Makefile index 3453f330e363..36f2727e1445 100644 --- a/arch/sparc/kernel/Makefile +++ b/arch/sparc/kernel/Makefile @@ -9,7 +9,7 @@ asflags-y := -ansi # Undefine sparc when processing vmlinux.lds - it is used # And teach CPP we are doing $(BITS) builds (for this case) CPPFLAGS_vmlinux.lds := -Usparc -m$(BITS) -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds ifdef CONFIG_FUNCTION_TRACER # Do not profile debug and lowlevel utilities diff --git a/arch/sparc/kernel/viohs.c b/arch/sparc/kernel/viohs.c index e27afd233bf5..8fb2e7ca5015 100644 --- a/arch/sparc/kernel/viohs.c +++ b/arch/sparc/kernel/viohs.c @@ -804,7 +804,7 @@ EXPORT_SYMBOL(vio_port_up); static void vio_port_timer(struct timer_list *t) { - struct vio_driver_state *vio = from_timer(vio, t, timer); + struct vio_driver_state *vio = timer_container_of(vio, t, timer); vio_port_up(vio); } diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c index 5226d2c52e6a..f292e0b4ff8b 100644 --- a/arch/um/drivers/vector_kern.c +++ b/arch/um/drivers/vector_kern.c @@ -1534,7 +1534,7 @@ static const struct net_device_ops vector_netdev_ops = { static void vector_timer_expire(struct timer_list *t) { - struct vector_private *vp = from_timer(vp, t, tl); + struct vector_private *vp = timer_container_of(vp, t, tl); vp->estats.tx_kicks++; napi_schedule(&vp->napi); diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h index ca2a519d53ab..24fdea6f88c3 100644 --- a/arch/um/include/asm/pgtable.h +++ b/arch/um/include/asm/pgtable.h @@ -314,7 +314,7 @@ extern pte_t *virt_to_pte(struct mm_struct *mm, unsigned long addr); ((swp_entry_t) { pte_val(pte_mkuptodate(pte)) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_get_bits(pte, _PAGE_SWP_EXCLUSIVE); } diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile index 4669db2aa9be..b8f4e9281599 100644 --- a/arch/um/kernel/Makefile +++ b/arch/um/kernel/Makefile @@ -12,7 +12,7 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START) \ -DELF_ARCH=$(LDS_ELF_ARCH) \ -DELF_FORMAT=$(LDS_ELF_FORMAT) \ $(LDS_EXTRA) -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \ physmem.o process.o ptrace.o reboot.o sigio.o \ diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 466283326630..741b229f0718 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2900,6 +2900,7 @@ static void intel_pmu_config_acr(int idx, u64 mask, u32 reload) { struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); int msr_b, msr_c; + int msr_offset; if (!mask && !cpuc->acr_cfg_b[idx]) return; @@ -2907,19 +2908,20 @@ static void intel_pmu_config_acr(int idx, u64 mask, u32 reload) if (idx < INTEL_PMC_IDX_FIXED) { msr_b = MSR_IA32_PMC_V6_GP0_CFG_B; msr_c = MSR_IA32_PMC_V6_GP0_CFG_C; + msr_offset = x86_pmu.addr_offset(idx, false); } else { msr_b = MSR_IA32_PMC_V6_FX0_CFG_B; msr_c = MSR_IA32_PMC_V6_FX0_CFG_C; - idx -= INTEL_PMC_IDX_FIXED; + msr_offset = x86_pmu.addr_offset(idx - INTEL_PMC_IDX_FIXED, false); } if (cpuc->acr_cfg_b[idx] != mask) { - wrmsrl(msr_b + x86_pmu.addr_offset(idx, false), mask); + wrmsrl(msr_b + msr_offset, mask); cpuc->acr_cfg_b[idx] = mask; } /* Only need to update the reload value when there is a valid config value. */ if (mask && cpuc->acr_cfg_c[idx] != reload) { - wrmsrl(msr_c + x86_pmu.addr_offset(idx, false), reload); + wrmsrl(msr_c + msr_offset, reload); cpuc->acr_cfg_c[idx] = reload; } } diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 774430c3abff..97954c936c54 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1561,7 +1561,7 @@ static inline pte_t pte_swp_mkexclusive(pte_t pte) return pte_set_flags(pte, _PAGE_SWP_EXCLUSIVE); } -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_flags(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/x86/include/asm/trace/fpu.h b/arch/x86/include/asm/trace/fpu.h index 0454d5e60e5d..721b408d9a67 100644 --- a/arch/x86/include/asm/trace/fpu.h +++ b/arch/x86/include/asm/trace/fpu.h @@ -44,16 +44,6 @@ DEFINE_EVENT(x86_fpu, x86_fpu_after_save, TP_ARGS(fpu) ); -DEFINE_EVENT(x86_fpu, x86_fpu_before_restore, - TP_PROTO(struct fpu *fpu), - TP_ARGS(fpu) -); - -DEFINE_EVENT(x86_fpu, x86_fpu_after_restore, - TP_PROTO(struct fpu *fpu), - TP_ARGS(fpu) -); - DEFINE_EVENT(x86_fpu, x86_fpu_regs_activated, TP_PROTO(struct fpu *fpu), TP_ARGS(fpu) @@ -64,11 +54,6 @@ DEFINE_EVENT(x86_fpu, x86_fpu_regs_deactivated, TP_ARGS(fpu) ); -DEFINE_EVENT(x86_fpu, x86_fpu_init_state, - TP_PROTO(struct fpu *fpu), - TP_ARGS(fpu) -); - DEFINE_EVENT(x86_fpu, x86_fpu_dropped, TP_PROTO(struct fpu *fpu), TP_ARGS(fpu) diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 99a783fd4691..0d2a6d953be9 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the linux kernel. # -extra-y += vmlinux.lds +always-$(KBUILD_BUILTIN) += vmlinux.lds CPPFLAGS_vmlinux.lds += -U$(UTS_MACHINE) diff --git a/arch/x86/kernel/ioport.c b/arch/x86/kernel/ioport.c index 6290dd120f5e..ff40f09ad911 100644 --- a/arch/x86/kernel/ioport.c +++ b/arch/x86/kernel/ioport.c @@ -33,8 +33,9 @@ void io_bitmap_share(struct task_struct *tsk) set_tsk_thread_flag(tsk, TIF_IO_BITMAP); } -static void task_update_io_bitmap(struct task_struct *tsk) +static void task_update_io_bitmap(void) { + struct task_struct *tsk = current; struct thread_struct *t = &tsk->thread; if (t->iopl_emul == 3 || t->io_bitmap) { @@ -54,7 +55,12 @@ void io_bitmap_exit(struct task_struct *tsk) struct io_bitmap *iobm = tsk->thread.io_bitmap; tsk->thread.io_bitmap = NULL; - task_update_io_bitmap(tsk); + /* + * Don't touch the TSS when invoked on a failed fork(). TSS + * reflects the state of @current and not the state of @tsk. + */ + if (tsk == current) + task_update_io_bitmap(); if (iobm && refcount_dec_and_test(&iobm->refcnt)) kfree(iobm); } @@ -192,8 +198,7 @@ SYSCALL_DEFINE1(iopl, unsigned int, level) } t->iopl_emul = level; - task_update_io_bitmap(current); - + task_update_io_bitmap(); return 0; } diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index c1d2dac72b9c..704883c21f3a 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -176,6 +176,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) frame->ret_addr = (unsigned long) ret_from_fork_asm; p->thread.sp = (unsigned long) fork_frame; p->thread.io_bitmap = NULL; + clear_tsk_thread_flag(p, TIF_IO_BITMAP); p->thread.iopl_warn = 0; memset(p->thread.ptrace_bps, 0, sizeof(p->thread.ptrace_bps)); @@ -464,6 +465,11 @@ void native_tss_update_io_bitmap(void) } else { struct io_bitmap *iobm = t->io_bitmap; + if (WARN_ON_ONCE(!iobm)) { + clear_thread_flag(TIF_IO_BITMAP); + native_tss_invalidate_io_bitmap(); + } + /* * Only copy bitmap data when the sequence number differs. The * update time is accounted to the incoming task. diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 18266cc3d98c..b014e6d229f9 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -299,3 +299,27 @@ struct smp_ops smp_ops = { .send_call_func_single_ipi = native_send_call_func_single_ipi, }; EXPORT_SYMBOL_GPL(smp_ops); + +int arch_cpu_rescan_dead_smt_siblings(void) +{ + enum cpuhp_smt_control old = cpu_smt_control; + int ret; + + /* + * If SMT has been disabled and SMT siblings are in HLT, bring them back + * online and offline them again so that they end up in MWAIT proper. + * + * Called with hotplug enabled. + */ + if (old != CPU_SMT_DISABLED && old != CPU_SMT_FORCE_DISABLED) + return 0; + + ret = cpuhp_smt_enable(); + if (ret) + return ret; + + ret = cpuhp_smt_disable(old); + + return ret; +} +EXPORT_SYMBOL_GPL(arch_cpu_rescan_dead_smt_siblings); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index fc78c2325fd2..58ede3fa6a75 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1244,6 +1244,10 @@ void play_dead_common(void) local_irq_disable(); } +/* + * We need to flush the caches before going to sleep, lest we have + * dirty data in our caches when we come back up. + */ void __noreturn mwait_play_dead(unsigned int eax_hint) { struct mwait_cpu_dead *md = this_cpu_ptr(&mwait_cpu_dead); @@ -1290,50 +1294,6 @@ void __noreturn mwait_play_dead(unsigned int eax_hint) } /* - * We need to flush the caches before going to sleep, lest we have - * dirty data in our caches when we come back up. - */ -static inline void mwait_play_dead_cpuid_hint(void) -{ - unsigned int eax, ebx, ecx, edx; - unsigned int highest_cstate = 0; - unsigned int highest_subcstate = 0; - int i; - - if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD || - boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) - return; - if (!this_cpu_has(X86_FEATURE_MWAIT)) - return; - if (!this_cpu_has(X86_FEATURE_CLFLUSH)) - return; - - eax = CPUID_LEAF_MWAIT; - ecx = 0; - native_cpuid(&eax, &ebx, &ecx, &edx); - - /* - * eax will be 0 if EDX enumeration is not valid. - * Initialized below to cstate, sub_cstate value when EDX is valid. - */ - if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED)) { - eax = 0; - } else { - edx >>= MWAIT_SUBSTATE_SIZE; - for (i = 0; i < 7 && edx; i++, edx >>= MWAIT_SUBSTATE_SIZE) { - if (edx & MWAIT_SUBSTATE_MASK) { - highest_cstate = i; - highest_subcstate = edx & MWAIT_SUBSTATE_MASK; - } - } - eax = (highest_cstate << MWAIT_SUBSTATE_SIZE) | - (highest_subcstate - 1); - } - - mwait_play_dead(eax); -} - -/* * Kick all "offline" CPUs out of mwait on kexec(). See comment in * mwait_play_dead(). */ @@ -1383,9 +1343,9 @@ void native_play_dead(void) play_dead_common(); tboot_shutdown(TB_SHUTDOWN_WFS); - mwait_play_dead_cpuid_hint(); - if (cpuidle_play_dead()) - hlt_play_dead(); + /* Below returns only on error. */ + cpuidle_play_dead(); + hlt_play_dead(); } #else /* ... !CONFIG_HOTPLUG_CPU */ diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index cbc84c6abc2e..4e06e2e89a8f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4896,12 +4896,16 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, { u64 error_code = PFERR_GUEST_FINAL_MASK; u8 level = PG_LEVEL_4K; + u64 direct_bits; u64 end; int r; if (!vcpu->kvm->arch.pre_fault_allowed) return -EOPNOTSUPP; + if (kvm_is_gfn_alias(vcpu->kvm, gpa_to_gfn(range->gpa))) + return -EINVAL; + /* * reload is efficient when called repeatedly, so we can do it on * every iteration. @@ -4910,15 +4914,18 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, if (r) return r; + direct_bits = 0; if (kvm_arch_has_private_mem(vcpu->kvm) && kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(range->gpa))) error_code |= PFERR_PRIVATE_ACCESS; + else + direct_bits = gfn_to_gpa(kvm_gfn_direct_bits(vcpu->kvm)); /* * Shadow paging uses GVA for kvm page fault, so restrict to * two-dimensional paging. */ - r = kvm_tdp_map_page(vcpu, range->gpa, error_code, &level); + r = kvm_tdp_map_page(vcpu, range->gpa | direct_bits, error_code, &level); if (r < 0) return r; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 5a69b657dae9..459c3b791fd4 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2871,6 +2871,33 @@ void __init sev_set_cpu_caps(void) } } +static bool is_sev_snp_initialized(void) +{ + struct sev_user_data_snp_status *status; + struct sev_data_snp_addr buf; + bool initialized = false; + int ret, error = 0; + + status = snp_alloc_firmware_page(GFP_KERNEL | __GFP_ZERO); + if (!status) + return false; + + buf.address = __psp_pa(status); + ret = sev_do_cmd(SEV_CMD_SNP_PLATFORM_STATUS, &buf, &error); + if (ret) { + pr_err("SEV: SNP_PLATFORM_STATUS failed ret=%d, fw_error=%d (%#x)\n", + ret, error, error); + goto out; + } + + initialized = !!status->state; + +out: + snp_free_firmware_page(status); + + return initialized; +} + void __init sev_hardware_setup(void) { unsigned int eax, ebx, ecx, edx, sev_asid_count, sev_es_asid_count; @@ -2975,6 +3002,14 @@ void __init sev_hardware_setup(void) sev_snp_supported = sev_snp_enabled && cc_platform_has(CC_ATTR_HOST_SEV_SNP); out: + if (sev_enabled) { + init_args.probe = true; + if (sev_platform_init(&init_args)) + sev_supported = sev_es_supported = sev_snp_supported = false; + else if (sev_snp_supported) + sev_snp_supported = is_sev_snp_initialized(); + } + if (boot_cpu_has(X86_FEATURE_SEV)) pr_info("SEV %s (ASIDs %u - %u)\n", sev_supported ? min_sev_asid <= max_sev_asid ? "enabled" : @@ -3001,15 +3036,6 @@ out: sev_supported_vmsa_features = 0; if (sev_es_debug_swap_enabled) sev_supported_vmsa_features |= SVM_SEV_FEAT_DEBUG_SWAP; - - if (!sev_enabled) - return; - - /* - * Do both SNP and SEV initialization at KVM module load. - */ - init_args.probe = true; - sev_platform_init(&init_args); } void sev_hardware_unsetup(void) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 38b33cdd4232..9b029bb29a16 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -1571,7 +1571,8 @@ out: static void cancel_evtchn_poll(struct timer_list *t) { - struct kvm_vcpu *vcpu = from_timer(vcpu, t, arch.xen.poll_timer); + struct kvm_vcpu *vcpu = timer_container_of(vcpu, t, + arch.xen.poll_timer); kvm_make_request(KVM_REQ_UNBLOCK, vcpu); kvm_vcpu_kick(vcpu); diff --git a/arch/x86/power/hibernate.c b/arch/x86/power/hibernate.c index a7c23f2a58c9..a2294c1649f6 100644 --- a/arch/x86/power/hibernate.c +++ b/arch/x86/power/hibernate.c @@ -192,7 +192,8 @@ out: int arch_resume_nosmt(void) { - int ret = 0; + int ret; + /* * We reached this while coming out of hibernation. This means * that SMT siblings are sleeping in hlt, as mwait is not safe @@ -206,18 +207,10 @@ int arch_resume_nosmt(void) * Called with hotplug disabled. */ cpu_hotplug_enable(); - if (cpu_smt_control == CPU_SMT_DISABLED || - cpu_smt_control == CPU_SMT_FORCE_DISABLED) { - enum cpuhp_smt_control old = cpu_smt_control; - - ret = cpuhp_smt_enable(); - if (ret) - goto out; - ret = cpuhp_smt_disable(old); - if (ret) - goto out; - } -out: + + ret = arch_cpu_rescan_dead_smt_siblings(); + cpu_hotplug_disable(); + return ret; } diff --git a/arch/xtensa/include/asm/pgtable.h b/arch/xtensa/include/asm/pgtable.h index cb1725c40e36..d62aa1c316fc 100644 --- a/arch/xtensa/include/asm/pgtable.h +++ b/arch/xtensa/include/asm/pgtable.h @@ -349,7 +349,7 @@ ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) -static inline int pte_swp_exclusive(pte_t pte) +static inline bool pte_swp_exclusive(pte_t pte) { return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; } diff --git a/arch/xtensa/kernel/Makefile b/arch/xtensa/kernel/Makefile index f28b8e3d717e..d3ef0407401f 100644 --- a/arch/xtensa/kernel/Makefile +++ b/arch/xtensa/kernel/Makefile @@ -3,7 +3,7 @@ # Makefile for the Linux/Xtensa kernel. # -extra-y := vmlinux.lds +always-$(KBUILD_BUILTIN) := vmlinux.lds obj-y := head.o align.o coprocessor.o entry.o irq.o platform.o process.o \ ptrace.o setup.o signal.o stacktrace.o syscall.o time.o traps.o \ diff --git a/arch/xtensa/platforms/iss/network.c b/arch/xtensa/platforms/iss/network.c index c6d8c62695e1..f0a63b2f85cc 100644 --- a/arch/xtensa/platforms/iss/network.c +++ b/arch/xtensa/platforms/iss/network.c @@ -338,7 +338,7 @@ static int iss_net_poll(struct iss_net_private *lp) static void iss_net_timer(struct timer_list *t) { - struct iss_net_private *lp = from_timer(lp, t, timer); + struct iss_net_private *lp = timer_container_of(lp, t, timer); iss_net_poll(lp); mod_timer(&lp->timer, jiffies + lp->timer_val); diff --git a/block/blk-core.c b/block/blk-core.c index b862c66018f2..fdac48aec5ef 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -381,7 +381,7 @@ static void blk_queue_usage_counter_release(struct percpu_ref *ref) static void blk_rq_timed_out_timer(struct timer_list *t) { - struct request_queue *q = from_timer(q, t, timeout); + struct request_queue *q = timer_container_of(q, t, timeout); kblockd_schedule_work(&q->timeout_work); } diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c index 42c1e0b9a68f..2f8fdecdd7a9 100644 --- a/block/blk-iolatency.c +++ b/block/blk-iolatency.c @@ -658,7 +658,8 @@ static const struct rq_qos_ops blkcg_iolatency_ops = { static void blkiolatency_timer_fn(struct timer_list *t) { - struct blk_iolatency *blkiolat = from_timer(blkiolat, t, timer); + struct blk_iolatency *blkiolat = timer_container_of(blkiolat, t, + timer); struct blkcg_gq *blkg; struct cgroup_subsys_state *pos_css; u64 now = blk_time_get_ns(); diff --git a/block/blk-stat.c b/block/blk-stat.c index 46449da856f8..682a8ddb1173 100644 --- a/block/blk-stat.c +++ b/block/blk-stat.c @@ -76,7 +76,7 @@ void blk_stat_add(struct request *rq, u64 now) static void blk_stat_timer_fn(struct timer_list *t) { - struct blk_stat_callback *cb = from_timer(cb, t, timer); + struct blk_stat_callback *cb = timer_container_of(cb, t, timer); unsigned int bucket; int cpu; diff --git a/block/blk-throttle.c b/block/blk-throttle.c index bd15357f23bd..397b6a410f9e 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1125,7 +1125,8 @@ static int throtl_select_dispatch(struct throtl_service_queue *parent_sq) */ static void throtl_pending_timer_fn(struct timer_list *t) { - struct throtl_service_queue *sq = from_timer(sq, t, pending_timer); + struct throtl_service_queue *sq = timer_container_of(sq, t, + pending_timer); struct throtl_grp *tg = sq_to_tg(sq); struct throtl_data *td = sq_to_td(sq); struct throtl_service_queue *parent_sq; diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c index 0f0f8452609a..4dba8405bd01 100644 --- a/block/kyber-iosched.c +++ b/block/kyber-iosched.c @@ -276,7 +276,7 @@ static void kyber_resize_domain(struct kyber_queue_data *kqd, static void kyber_timer_fn(struct timer_list *t) { - struct kyber_queue_data *kqd = from_timer(kqd, t, timer); + struct kyber_queue_data *kqd = timer_container_of(kqd, t, timer); unsigned int sched_domain; int cpu; bool bad = false; diff --git a/crypto/hkdf.c b/crypto/hkdf.c index f24c2a8d4df9..82d1b32ca6ce 100644 --- a/crypto/hkdf.c +++ b/crypto/hkdf.c @@ -566,7 +566,7 @@ static int __init crypto_hkdf_module_init(void) static void __exit crypto_hkdf_module_exit(void) {} -module_init(crypto_hkdf_module_init); +late_initcall(crypto_hkdf_module_init); module_exit(crypto_hkdf_module_exit); MODULE_LICENSE("GPL"); diff --git a/drivers/accel/qaic/qaic_timesync.c b/drivers/accel/qaic/qaic_timesync.c index 972833fabcfc..3fac540f8e03 100644 --- a/drivers/accel/qaic/qaic_timesync.c +++ b/drivers/accel/qaic/qaic_timesync.c @@ -129,7 +129,7 @@ static void qaic_timesync_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_resu static void qaic_timesync_timer(struct timer_list *t) { - struct mqts_dev *mqtsdev = from_timer(mqtsdev, t, timer); + struct mqts_dev *mqtsdev = timer_container_of(mqtsdev, t, timer); struct qts_host_time_sync_msg_data *sync_msg; u64 device_qtimer_us; u64 device_qtimer; diff --git a/drivers/acpi/acpi_pad.c b/drivers/acpi/acpi_pad.c index 6f8bbe1247a5..c9a0bcaba2e4 100644 --- a/drivers/acpi/acpi_pad.c +++ b/drivers/acpi/acpi_pad.c @@ -33,7 +33,7 @@ static DEFINE_MUTEX(isolated_cpus_lock); static DEFINE_MUTEX(round_robin_lock); -static unsigned long power_saving_mwait_eax; +static unsigned int power_saving_mwait_eax; static unsigned char tsc_detected_unstable; static unsigned char tsc_marked_unstable; diff --git a/drivers/acpi/apei/einj-core.c b/drivers/acpi/apei/einj-core.c index fea11a35eea3..9b041415a9d0 100644 --- a/drivers/acpi/apei/einj-core.c +++ b/drivers/acpi/apei/einj-core.c @@ -883,19 +883,16 @@ static int __init einj_init(void) } einj_dev = faux_device_create("acpi-einj", NULL, &einj_device_ops); - if (!einj_dev) - return -ENODEV; - einj_initialized = true; + if (einj_dev) + einj_initialized = true; return 0; } static void __exit einj_exit(void) { - if (einj_initialized) - faux_device_destroy(einj_dev); - + faux_device_destroy(einj_dev); } module_init(einj_init); diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 0f3c663c1b0a..f0584ccad451 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -1138,7 +1138,7 @@ static void ghes_add_timer(struct ghes *ghes) static void ghes_poll_func(struct timer_list *t) { - struct ghes *ghes = from_timer(ghes, t, timer); + struct ghes *ghes = timer_container_of(ghes, t, timer); unsigned long flags; spin_lock_irqsave(&ghes_notify_lock_irq, flags); diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index a9ae2fd62863..6b649031808f 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -476,7 +476,7 @@ bool cppc_allow_fast_switch(void) struct cpc_desc *cpc_ptr; int cpu; - for_each_possible_cpu(cpu) { + for_each_present_cpu(cpu) { cpc_ptr = per_cpu(cpc_desc_ptr, cpu); desired_reg = &cpc_ptr->cpc_regs[DESIRED_PERF]; if (!CPC_IN_SYSTEM_MEMORY(desired_reg) && diff --git a/drivers/acpi/ec.c b/drivers/acpi/ec.c index 6f4203716b53..75c7db8b156a 100644 --- a/drivers/acpi/ec.c +++ b/drivers/acpi/ec.c @@ -23,8 +23,10 @@ #include <linux/delay.h> #include <linux/interrupt.h> #include <linux/list.h> +#include <linux/printk.h> #include <linux/spinlock.h> #include <linux/slab.h> +#include <linux/string.h> #include <linux/suspend.h> #include <linux/acpi.h> #include <linux/dmi.h> @@ -2031,6 +2033,21 @@ void __init acpi_ec_ecdt_probe(void) goto out; } + if (!strstarts(ecdt_ptr->id, "\\")) { + /* + * The ECDT table on some MSI notebooks contains invalid data, together + * with an empty ID string (""). + * + * Section 5.2.15 of the ACPI specification requires the ID string to be + * a "fully qualified reference to the (...) embedded controller device", + * so this string always has to start with a backslash. + * + * By verifying this we can avoid such faulty ECDT tables in a safe way. + */ + pr_err(FW_BUG "Ignoring ECDT due to invalid ID string \"%s\"\n", ecdt_ptr->id); + goto out; + } + ec = acpi_ec_alloc(); if (!ec) goto out; diff --git a/drivers/acpi/internal.h b/drivers/acpi/internal.h index 00910ccd7eda..e2781864fdce 100644 --- a/drivers/acpi/internal.h +++ b/drivers/acpi/internal.h @@ -175,6 +175,12 @@ bool processor_physically_present(acpi_handle handle); static inline void acpi_early_processor_control_setup(void) {} #endif +#ifdef CONFIG_ACPI_PROCESSOR_CSTATE +void acpi_idle_rescan_dead_smt_siblings(void); +#else +static inline void acpi_idle_rescan_dead_smt_siblings(void) {} +#endif + /* -------------------------------------------------------------------------- Embedded Controller -------------------------------------------------------------------------- */ diff --git a/drivers/acpi/processor_driver.c b/drivers/acpi/processor_driver.c index 3b281bc1e73c..65e779be64ff 100644 --- a/drivers/acpi/processor_driver.c +++ b/drivers/acpi/processor_driver.c @@ -279,6 +279,9 @@ static int __init acpi_processor_driver_init(void) * after acpi_cppc_processor_probe() has been called for all online CPUs */ acpi_processor_init_invariance_cppc(); + + acpi_idle_rescan_dead_smt_siblings(); + return 0; err: driver_unregister(&acpi_processor_driver); diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index e2febca2ec13..2c2dc559e0f8 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -24,6 +24,8 @@ #include <acpi/processor.h> #include <linux/context_tracking.h> +#include "internal.h" + /* * Include the apic definitions for x86 to have the APIC timer related defines * available also for UP (on SMP it gets magically included via linux/smp.h). @@ -55,6 +57,12 @@ struct cpuidle_driver acpi_idle_driver = { }; #ifdef CONFIG_ACPI_PROCESSOR_CSTATE +void acpi_idle_rescan_dead_smt_siblings(void) +{ + if (cpuidle_get_driver() == &acpi_idle_driver) + arch_cpu_rescan_dead_smt_siblings(); +} + static DEFINE_PER_CPU(struct acpi_processor_cx * [CPUIDLE_STATE_MAX], acpi_cstate); diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c index 7d59c6c9185f..b1ab192d7a08 100644 --- a/drivers/acpi/resource.c +++ b/drivers/acpi/resource.c @@ -667,6 +667,13 @@ static const struct dmi_system_id irq1_edge_low_force_override[] = { }, }, { + /* MACHENIKE L16P/L16P */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "MACHENIKE"), + DMI_MATCH(DMI_BOARD_NAME, "L16P"), + }, + }, + { /* * TongFang GM5HG0A in case of the SKIKK Vanaheim relabel the * board-name is changed, so check OEM strings instead. Note diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c index 22afa4ff860d..4e9c82f36df1 100644 --- a/drivers/ata/libahci.c +++ b/drivers/ata/libahci.c @@ -1034,7 +1034,7 @@ static void ahci_sw_activity(struct ata_link *link) static void ahci_sw_activity_blink(struct timer_list *t) { - struct ahci_em_priv *emp = from_timer(emp, t, timer); + struct ahci_em_priv *emp = timer_container_of(emp, t, timer); struct ata_link *link = emp->link; struct ata_port *ap = link->ap; diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index c11d8e634bf7..1f90fcd2b3c4 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -860,7 +860,7 @@ static unsigned int ata_eh_nr_in_flight(struct ata_port *ap) void ata_eh_fastdrain_timerfn(struct timer_list *t) { - struct ata_port *ap = from_timer(ap, t, fastdrain_timer); + struct ata_port *ap = timer_container_of(ap, t, fastdrain_timer); unsigned long flags; unsigned int cnt; diff --git a/drivers/atm/idt77252.c b/drivers/atm/idt77252.c index a876024d8a05..1206ab764ba9 100644 --- a/drivers/atm/idt77252.c +++ b/drivers/atm/idt77252.c @@ -1531,7 +1531,7 @@ idt77252_tx(struct idt77252_dev *card) static void tst_timer(struct timer_list *t) { - struct idt77252_dev *card = from_timer(card, t, tst_timer); + struct idt77252_dev *card = timer_container_of(card, t, tst_timer); unsigned long base, idle, jump; unsigned long flags; u32 pc; @@ -2070,7 +2070,7 @@ idt77252_rate_logindex(struct idt77252_dev *card, int pcr) static void idt77252_est_timer(struct timer_list *t) { - struct rate_estimator *est = from_timer(est, t, timer); + struct rate_estimator *est = timer_container_of(est, t, timer); struct vc_map *vc = est->vc; struct idt77252_dev *card = vc->card; unsigned long flags; diff --git a/drivers/atm/lanai.c b/drivers/atm/lanai.c index 00fe25b5b6a3..2a1fe3080712 100644 --- a/drivers/atm/lanai.c +++ b/drivers/atm/lanai.c @@ -1758,7 +1758,7 @@ static void iter_dequeue(struct lanai_dev *lanai, vci_t vci) static void lanai_timed_poll(struct timer_list *t) { - struct lanai_dev *lanai = from_timer(lanai, t, timer); + struct lanai_dev *lanai = timer_container_of(lanai, t, timer); #ifndef DEBUG_RW unsigned long flags; #ifdef USE_POWERDOWN diff --git a/drivers/auxdisplay/line-display.c b/drivers/auxdisplay/line-display.c index b6808c4f89b6..8590a4cd21e0 100644 --- a/drivers/auxdisplay/line-display.c +++ b/drivers/auxdisplay/line-display.c @@ -40,7 +40,7 @@ */ static void linedisp_scroll(struct timer_list *t) { - struct linedisp *linedisp = from_timer(linedisp, t, timer); + struct linedisp *linedisp = timer_container_of(linedisp, t, timer); unsigned int i, ch = linedisp->scroll_pos; unsigned int num_chars = linedisp->num_chars; diff --git a/drivers/base/faux.c b/drivers/base/faux.c index 9054d346bd7f..f5fbda0a9a44 100644 --- a/drivers/base/faux.c +++ b/drivers/base/faux.c @@ -86,6 +86,7 @@ static struct device_driver faux_driver = { .name = "faux_driver", .bus = &faux_bus_type, .probe_type = PROBE_FORCE_SYNCHRONOUS, + .suppress_bind_attrs = true, }; static void faux_device_release(struct device *dev) @@ -169,7 +170,7 @@ struct faux_device *faux_device_create_with_groups(const char *name, * successful is almost impossible to determine by the caller. */ if (!dev->driver) { - dev_err(dev, "probe did not succeed, tearing down the device\n"); + dev_dbg(dev, "probe did not succeed, tearing down the device\n"); faux_device_destroy(faux_dev); faux_dev = NULL; } diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c index 77c7a99f0870..eebe699fdf4f 100644 --- a/drivers/base/power/main.c +++ b/drivers/base/power/main.c @@ -513,7 +513,7 @@ struct dpm_watchdog { */ static void dpm_watchdog_handler(struct timer_list *t) { - struct dpm_watchdog *wd = from_timer(wd, t, timer); + struct dpm_watchdog *wd = timer_container_of(wd, t, timer); struct timer_list *timer = &wd->timer; unsigned int time_left; diff --git a/drivers/base/power/wakeup.c b/drivers/base/power/wakeup.c index f7c96a3bf719..d1283ff1080b 100644 --- a/drivers/base/power/wakeup.c +++ b/drivers/base/power/wakeup.c @@ -759,7 +759,7 @@ EXPORT_SYMBOL_GPL(pm_relax); */ static void pm_wakeup_timer_fn(struct timer_list *t) { - struct wakeup_source *ws = from_timer(ws, t, timer); + struct wakeup_source *ws = timer_container_of(ws, t, timer); unsigned long flags; spin_lock_irqsave(&ws->lock, flags); diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c index 92b06d1de4cc..50cc90f6ab35 100644 --- a/drivers/block/aoe/aoecmd.c +++ b/drivers/block/aoe/aoecmd.c @@ -745,7 +745,7 @@ rexmit_timer(struct timer_list *timer) int utgts; /* number of aoetgt descriptors (not slots) */ int since; - d = from_timer(d, timer, timer); + d = timer_container_of(d, timer, timer); spin_lock_irqsave(&d->lock, flags); diff --git a/drivers/block/aoe/aoedev.c b/drivers/block/aoe/aoedev.c index 141b2a0e03f2..bba05f0c5bbd 100644 --- a/drivers/block/aoe/aoedev.c +++ b/drivers/block/aoe/aoedev.c @@ -149,7 +149,7 @@ dummy_timer(struct timer_list *t) { struct aoedev *d; - d = from_timer(d, t, timer); + d = timer_container_of(d, t, timer); if (d->flags & DEVFL_TKILL) return; d->timer.expires = jiffies + HZ; diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c index ced2cc5f46f2..52724b79be30 100644 --- a/drivers/block/drbd/drbd_main.c +++ b/drivers/block/drbd/drbd_main.c @@ -3591,7 +3591,8 @@ int drbd_md_test_flag(struct drbd_backing_dev *bdev, int flag) static void md_sync_timer_fn(struct timer_list *t) { - struct drbd_device *device = from_timer(device, t, md_sync_timer); + struct drbd_device *device = timer_container_of(device, t, + md_sync_timer); drbd_device_post_work(device, MD_SYNC); } diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c index 380e6584a4ee..d15826f6ee81 100644 --- a/drivers/block/drbd/drbd_req.c +++ b/drivers/block/drbd/drbd_req.c @@ -1699,7 +1699,8 @@ static bool net_timeout_reached(struct drbd_request *net_req, void request_timer_fn(struct timer_list *t) { - struct drbd_device *device = from_timer(device, t, request_timer); + struct drbd_device *device = timer_container_of(device, t, + request_timer); struct drbd_connection *connection = first_peer_device(device)->connection; struct drbd_request *req_read, *req_write, *req_peer; /* oldest request */ struct net_conf *nc; diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c index 4352a50fbb3f..a6ea737b3b71 100644 --- a/drivers/block/drbd/drbd_worker.c +++ b/drivers/block/drbd/drbd_worker.c @@ -442,7 +442,8 @@ int w_resync_timer(struct drbd_work *w, int cancel) void resync_timer_fn(struct timer_list *t) { - struct drbd_device *device = from_timer(device, t, resync_timer); + struct drbd_device *device = timer_container_of(device, t, + resync_timer); drbd_queue_work_if_unqueued( &first_peer_device(device)->connection->sender_work, @@ -1698,7 +1699,8 @@ void drbd_rs_controller_reset(struct drbd_peer_device *peer_device) void start_resync_timer_fn(struct timer_list *t) { - struct drbd_device *device = from_timer(device, t, start_resync_timer); + struct drbd_device *device = timer_container_of(device, t, + start_resync_timer); drbd_device_post_work(device, RS_START); } diff --git a/drivers/block/swim3.c b/drivers/block/swim3.c index ee6cade70222..01f7aef3fcfb 100644 --- a/drivers/block/swim3.c +++ b/drivers/block/swim3.c @@ -555,7 +555,7 @@ static void act(struct floppy_state *fs) static void scan_timeout(struct timer_list *t) { - struct floppy_state *fs = from_timer(fs, t, timeout); + struct floppy_state *fs = timer_container_of(fs, t, timeout); struct swim3 __iomem *sw = fs->swim3; unsigned long flags; @@ -579,7 +579,7 @@ static void scan_timeout(struct timer_list *t) static void seek_timeout(struct timer_list *t) { - struct floppy_state *fs = from_timer(fs, t, timeout); + struct floppy_state *fs = timer_container_of(fs, t, timeout); struct swim3 __iomem *sw = fs->swim3; unsigned long flags; @@ -598,7 +598,7 @@ static void seek_timeout(struct timer_list *t) static void settle_timeout(struct timer_list *t) { - struct floppy_state *fs = from_timer(fs, t, timeout); + struct floppy_state *fs = timer_container_of(fs, t, timeout); struct swim3 __iomem *sw = fs->swim3; unsigned long flags; @@ -627,7 +627,7 @@ static void settle_timeout(struct timer_list *t) static void xfer_timeout(struct timer_list *t) { - struct floppy_state *fs = from_timer(fs, t, timeout); + struct floppy_state *fs = timer_container_of(fs, t, timeout); struct swim3 __iomem *sw = fs->swim3; struct dbdma_regs __iomem *dr = fs->dma; unsigned long flags; diff --git a/drivers/bluetooth/bluecard_cs.c b/drivers/bluetooth/bluecard_cs.c index 1c7f89e134b3..1e3a56e9b139 100644 --- a/drivers/bluetooth/bluecard_cs.c +++ b/drivers/bluetooth/bluecard_cs.c @@ -158,7 +158,7 @@ static void bluecard_detach(struct pcmcia_device *p_dev); static void bluecard_activity_led_timeout(struct timer_list *t) { - struct bluecard_info *info = from_timer(info, t, timer); + struct bluecard_info *info = timer_container_of(info, t, timer); unsigned int iobase = info->p_dev->resource[0]->start; if (test_bit(CARD_ACTIVITY, &(info->hw_state))) { diff --git a/drivers/bluetooth/btintel_pcie.c b/drivers/bluetooth/btintel_pcie.c index 50fe17f1e1d1..563165c5efae 100644 --- a/drivers/bluetooth/btintel_pcie.c +++ b/drivers/bluetooth/btintel_pcie.c @@ -396,8 +396,13 @@ static int btintel_pcie_submit_rx(struct btintel_pcie_data *data) static int btintel_pcie_start_rx(struct btintel_pcie_data *data) { int i, ret; + struct rxq *rxq = &data->rxq; + + /* Post (BTINTEL_PCIE_RX_DESCS_COUNT - 3) buffers to overcome the + * hardware issues leading to race condition at the firmware. + */ - for (i = 0; i < BTINTEL_PCIE_RX_MAX_QUEUE; i++) { + for (i = 0; i < rxq->count - 3; i++) { ret = btintel_pcie_submit_rx(data); if (ret) return ret; @@ -1782,8 +1787,8 @@ static int btintel_pcie_alloc(struct btintel_pcie_data *data) * + size of index * Number of queues(2) * type of index array(4) * + size of context information */ - total = (sizeof(struct tfd) + sizeof(struct urbd0) + sizeof(struct frbd) - + sizeof(struct urbd1)) * BTINTEL_DESCS_COUNT; + total = (sizeof(struct tfd) + sizeof(struct urbd0)) * BTINTEL_PCIE_TX_DESCS_COUNT; + total += (sizeof(struct frbd) + sizeof(struct urbd1)) * BTINTEL_PCIE_RX_DESCS_COUNT; /* Add the sum of size of index array and size of ci struct */ total += (sizeof(u16) * BTINTEL_PCIE_NUM_QUEUES * 4) + sizeof(struct ctx_info); @@ -1808,36 +1813,36 @@ static int btintel_pcie_alloc(struct btintel_pcie_data *data) data->dma_v_addr = v_addr; /* Setup descriptor count */ - data->txq.count = BTINTEL_DESCS_COUNT; - data->rxq.count = BTINTEL_DESCS_COUNT; + data->txq.count = BTINTEL_PCIE_TX_DESCS_COUNT; + data->rxq.count = BTINTEL_PCIE_RX_DESCS_COUNT; /* Setup tfds */ data->txq.tfds_p_addr = p_addr; data->txq.tfds = v_addr; - p_addr += (sizeof(struct tfd) * BTINTEL_DESCS_COUNT); - v_addr += (sizeof(struct tfd) * BTINTEL_DESCS_COUNT); + p_addr += (sizeof(struct tfd) * BTINTEL_PCIE_TX_DESCS_COUNT); + v_addr += (sizeof(struct tfd) * BTINTEL_PCIE_TX_DESCS_COUNT); /* Setup urbd0 */ data->txq.urbd0s_p_addr = p_addr; data->txq.urbd0s = v_addr; - p_addr += (sizeof(struct urbd0) * BTINTEL_DESCS_COUNT); - v_addr += (sizeof(struct urbd0) * BTINTEL_DESCS_COUNT); + p_addr += (sizeof(struct urbd0) * BTINTEL_PCIE_TX_DESCS_COUNT); + v_addr += (sizeof(struct urbd0) * BTINTEL_PCIE_TX_DESCS_COUNT); /* Setup FRBD*/ data->rxq.frbds_p_addr = p_addr; data->rxq.frbds = v_addr; - p_addr += (sizeof(struct frbd) * BTINTEL_DESCS_COUNT); - v_addr += (sizeof(struct frbd) * BTINTEL_DESCS_COUNT); + p_addr += (sizeof(struct frbd) * BTINTEL_PCIE_RX_DESCS_COUNT); + v_addr += (sizeof(struct frbd) * BTINTEL_PCIE_RX_DESCS_COUNT); /* Setup urbd1 */ data->rxq.urbd1s_p_addr = p_addr; data->rxq.urbd1s = v_addr; - p_addr += (sizeof(struct urbd1) * BTINTEL_DESCS_COUNT); - v_addr += (sizeof(struct urbd1) * BTINTEL_DESCS_COUNT); + p_addr += (sizeof(struct urbd1) * BTINTEL_PCIE_RX_DESCS_COUNT); + v_addr += (sizeof(struct urbd1) * BTINTEL_PCIE_RX_DESCS_COUNT); /* Setup data buffers for txq */ err = btintel_pcie_setup_txq_bufs(data, &data->txq); diff --git a/drivers/bluetooth/btintel_pcie.h b/drivers/bluetooth/btintel_pcie.h index 21b964b15c1c..7dad4523236c 100644 --- a/drivers/bluetooth/btintel_pcie.h +++ b/drivers/bluetooth/btintel_pcie.h @@ -154,8 +154,11 @@ enum msix_mbox_int_causes { /* Default interrupt timeout in msec */ #define BTINTEL_DEFAULT_INTR_TIMEOUT_MS 3000 -/* The number of descriptors in TX/RX queues */ -#define BTINTEL_DESCS_COUNT 16 +/* The number of descriptors in TX queues */ +#define BTINTEL_PCIE_TX_DESCS_COUNT 32 + +/* The number of descriptors in RX queues */ +#define BTINTEL_PCIE_RX_DESCS_COUNT 64 /* Number of Queue for TX and RX * It indicates the index of the IA(Index Array) @@ -177,9 +180,6 @@ enum { /* Doorbell vector for TFD */ #define BTINTEL_PCIE_TX_DB_VEC 0 -/* Number of pending RX requests for downlink */ -#define BTINTEL_PCIE_RX_MAX_QUEUE 6 - /* Doorbell vector for FRBD */ #define BTINTEL_PCIE_RX_DB_VEC 513 diff --git a/drivers/bluetooth/btnxpuart.c b/drivers/bluetooth/btnxpuart.c index 6b13feed06df..1088db6056a4 100644 --- a/drivers/bluetooth/btnxpuart.c +++ b/drivers/bluetooth/btnxpuart.c @@ -467,7 +467,7 @@ static void ps_work_func(struct work_struct *work) static void ps_timeout_func(struct timer_list *t) { - struct ps_data *data = from_timer(data, t, ps_timer); + struct ps_data *data = timer_container_of(data, t, ps_timer); struct hci_dev *hdev = data->hdev; struct btnxpuart_dev *nxpdev = hci_get_drvdata(hdev); diff --git a/drivers/bluetooth/hci_bcsp.c b/drivers/bluetooth/hci_bcsp.c index 610d0e3c36d4..664d82d1e613 100644 --- a/drivers/bluetooth/hci_bcsp.c +++ b/drivers/bluetooth/hci_bcsp.c @@ -688,7 +688,7 @@ static int bcsp_recv(struct hci_uart *hu, const void *data, int count) /* Arrange to retransmit all messages in the relq. */ static void bcsp_timed_event(struct timer_list *t) { - struct bcsp_struct *bcsp = from_timer(bcsp, t, tbcsp); + struct bcsp_struct *bcsp = timer_container_of(bcsp, t, tbcsp); struct hci_uart *hu = bcsp->hu; struct sk_buff *skb; unsigned long flags; diff --git a/drivers/bluetooth/hci_h5.c b/drivers/bluetooth/hci_h5.c index edafa228bf83..d0d4420c1a0f 100644 --- a/drivers/bluetooth/hci_h5.c +++ b/drivers/bluetooth/hci_h5.c @@ -149,7 +149,7 @@ static void h5_timed_event(struct timer_list *t) { const unsigned char sync_req[] = { 0x01, 0x7e }; unsigned char conf_req[3] = { 0x03, 0xfc }; - struct h5 *h5 = from_timer(h5, t, timer); + struct h5 *h5 = timer_container_of(h5, t, timer); struct hci_uart *hu = h5->hu; struct sk_buff *skb; unsigned long flags; diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c index a2dc39c005f4..5fe5879881f5 100644 --- a/drivers/bluetooth/hci_qca.c +++ b/drivers/bluetooth/hci_qca.c @@ -474,7 +474,7 @@ static void qca_wq_serial_tx_clock_vote_off(struct work_struct *work) static void hci_ibs_tx_idle_timeout(struct timer_list *t) { - struct qca_data *qca = from_timer(qca, t, tx_idle_timer); + struct qca_data *qca = timer_container_of(qca, t, tx_idle_timer); struct hci_uart *hu = qca->hu; unsigned long flags; @@ -507,7 +507,7 @@ static void hci_ibs_tx_idle_timeout(struct timer_list *t) static void hci_ibs_wake_retrans_timeout(struct timer_list *t) { - struct qca_data *qca = from_timer(qca, t, wake_retrans_timer); + struct qca_data *qca = timer_container_of(qca, t, wake_retrans_timer); struct hci_uart *hu = qca->hu; unsigned long flags, retrans_delay; bool retransmit = false; diff --git a/drivers/bus/mhi/host/pci_generic.c b/drivers/bus/mhi/host/pci_generic.c index a4a62429c784..589cb6722316 100644 --- a/drivers/bus/mhi/host/pci_generic.c +++ b/drivers/bus/mhi/host/pci_generic.c @@ -1171,7 +1171,8 @@ err_try_reset: static void health_check(struct timer_list *t) { - struct mhi_pci_device *mhi_pdev = from_timer(mhi_pdev, t, health_check_timer); + struct mhi_pci_device *mhi_pdev = timer_container_of(mhi_pdev, t, + health_check_timer); struct mhi_controller *mhi_cntrl = &mhi_pdev->mhi_cntrl; if (!test_bit(MHI_PCI_DEV_STARTED, &mhi_pdev->status) || diff --git a/drivers/char/hw_random/xgene-rng.c b/drivers/char/hw_random/xgene-rng.c index a1a751074f7e..709a36507145 100644 --- a/drivers/char/hw_random/xgene-rng.c +++ b/drivers/char/hw_random/xgene-rng.c @@ -88,7 +88,7 @@ struct xgene_rng_dev { static void xgene_rng_expired_timer(struct timer_list *t) { - struct xgene_rng_dev *ctx = from_timer(ctx, t, failure_timer); + struct xgene_rng_dev *ctx = timer_container_of(ctx, t, failure_timer); /* Clear failure counter as timer expired */ disable_irq(ctx->irq); diff --git a/drivers/char/ipmi/bt-bmc.c b/drivers/char/ipmi/bt-bmc.c index 77146b5c762b..a179d4797011 100644 --- a/drivers/char/ipmi/bt-bmc.c +++ b/drivers/char/ipmi/bt-bmc.c @@ -347,7 +347,7 @@ static const struct file_operations bt_bmc_fops = { static void poll_timer(struct timer_list *t) { - struct bt_bmc *bt_bmc = from_timer(bt_bmc, t, poll_timer); + struct bt_bmc *bt_bmc = timer_container_of(bt_bmc, t, poll_timer); bt_bmc->poll_timer.expires += msecs_to_jiffies(500); wake_up(&bt_bmc->queue); diff --git a/drivers/char/ipmi/ipmi_si_intf.c b/drivers/char/ipmi/ipmi_si_intf.c index 7fe891783a37..bb42dfe1c6a8 100644 --- a/drivers/char/ipmi/ipmi_si_intf.c +++ b/drivers/char/ipmi/ipmi_si_intf.c @@ -1076,7 +1076,8 @@ static void set_need_watch(void *send_info, unsigned int watch_mask) static void smi_timeout(struct timer_list *t) { - struct smi_info *smi_info = from_timer(smi_info, t, si_timer); + struct smi_info *smi_info = timer_container_of(smi_info, t, + si_timer); enum si_sm_result smi_result; unsigned long flags; unsigned long jiffies_now; diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c index 5bf038e620c7..1bc42830444d 100644 --- a/drivers/char/ipmi/ipmi_ssif.c +++ b/drivers/char/ipmi/ipmi_ssif.c @@ -539,7 +539,8 @@ static void start_resend(struct ssif_info *ssif_info); static void retry_timeout(struct timer_list *t) { - struct ssif_info *ssif_info = from_timer(ssif_info, t, retry_timer); + struct ssif_info *ssif_info = timer_container_of(ssif_info, t, + retry_timer); unsigned long oflags, *flags; bool waiting, resend; @@ -563,7 +564,8 @@ static void retry_timeout(struct timer_list *t) static void watch_timeout(struct timer_list *t) { - struct ssif_info *ssif_info = from_timer(ssif_info, t, watch_timer); + struct ssif_info *ssif_info = timer_container_of(ssif_info, t, + watch_timer); unsigned long oflags, *flags; if (ssif_info->stopping) diff --git a/drivers/char/ipmi/ssif_bmc.c b/drivers/char/ipmi/ssif_bmc.c index e4bd74585d4d..7a52e3ea49ed 100644 --- a/drivers/char/ipmi/ssif_bmc.c +++ b/drivers/char/ipmi/ssif_bmc.c @@ -297,7 +297,8 @@ static void complete_response(struct ssif_bmc_ctx *ssif_bmc) static void response_timeout(struct timer_list *t) { - struct ssif_bmc_ctx *ssif_bmc = from_timer(ssif_bmc, t, response_timer); + struct ssif_bmc_ctx *ssif_bmc = timer_container_of(ssif_bmc, t, + response_timer); unsigned long flags; spin_lock_irqsave(&ssif_bmc->lock, flags); diff --git a/drivers/char/tpm/tpm-dev-common.c b/drivers/char/tpm/tpm-dev-common.c index 11deaf538e87..f2a5e09257dd 100644 --- a/drivers/char/tpm/tpm-dev-common.c +++ b/drivers/char/tpm/tpm-dev-common.c @@ -91,7 +91,7 @@ out: static void user_reader_timeout(struct timer_list *t) { - struct file_priv *priv = from_timer(priv, t, user_read_timer); + struct file_priv *priv = timer_container_of(priv, t, user_read_timer); pr_warn("TPM user space timeout is deprecated (pid=%d)\n", task_tgid_nr(current)); diff --git a/drivers/comedi/drivers/comedi_test.c b/drivers/comedi/drivers/comedi_test.c index da17d891f0e5..9747e6d1f6eb 100644 --- a/drivers/comedi/drivers/comedi_test.c +++ b/drivers/comedi/drivers/comedi_test.c @@ -197,7 +197,8 @@ static unsigned short fake_waveform(struct comedi_device *dev, */ static void waveform_ai_timer(struct timer_list *t) { - struct waveform_private *devpriv = from_timer(devpriv, t, ai_timer); + struct waveform_private *devpriv = timer_container_of(devpriv, t, + ai_timer); struct comedi_device *dev = devpriv->dev; struct comedi_subdevice *s = dev->read_subdev; struct comedi_async *async = s->async; @@ -444,7 +445,8 @@ static int waveform_ai_insn_read(struct comedi_device *dev, */ static void waveform_ao_timer(struct timer_list *t) { - struct waveform_private *devpriv = from_timer(devpriv, t, ao_timer); + struct waveform_private *devpriv = timer_container_of(devpriv, t, + ao_timer); struct comedi_device *dev = devpriv->dev; struct comedi_subdevice *s = dev->write_subdev; struct comedi_async *async = s->async; diff --git a/drivers/comedi/drivers/das16.c b/drivers/comedi/drivers/das16.c index f5ca6c0d4d0c..1f85572c21b4 100644 --- a/drivers/comedi/drivers/das16.c +++ b/drivers/comedi/drivers/das16.c @@ -517,7 +517,8 @@ static void das16_interrupt(struct comedi_device *dev) static void das16_timer_interrupt(struct timer_list *t) { - struct das16_private_struct *devpriv = from_timer(devpriv, t, timer); + struct das16_private_struct *devpriv = timer_container_of(devpriv, t, + timer); struct comedi_device *dev = devpriv->dev; unsigned long flags; diff --git a/drivers/comedi/drivers/jr3_pci.c b/drivers/comedi/drivers/jr3_pci.c index 75dce1ff2419..61792d940a3d 100644 --- a/drivers/comedi/drivers/jr3_pci.c +++ b/drivers/comedi/drivers/jr3_pci.c @@ -562,7 +562,8 @@ jr3_pci_poll_subdevice(struct comedi_subdevice *s) static void jr3_pci_poll_dev(struct timer_list *t) { - struct jr3_pci_dev_private *devpriv = from_timer(devpriv, t, timer); + struct jr3_pci_dev_private *devpriv = timer_container_of(devpriv, t, + timer); struct comedi_device *dev = devpriv->dev; struct jr3_pci_subdev_private *spriv; struct comedi_subdevice *s; diff --git a/drivers/cpufreq/powernv-cpufreq.c b/drivers/cpufreq/powernv-cpufreq.c index afe5abf89d33..a8943e2a93be 100644 --- a/drivers/cpufreq/powernv-cpufreq.c +++ b/drivers/cpufreq/powernv-cpufreq.c @@ -667,7 +667,8 @@ static inline void queue_gpstate_timer(struct global_pstate_info *gpstates) */ static void gpstate_timer_handler(struct timer_list *t) { - struct global_pstate_info *gpstates = from_timer(gpstates, t, timer); + struct global_pstate_info *gpstates = timer_container_of(gpstates, t, + timer); struct cpufreq_policy *policy = gpstates->policy; int gpstate_idx, lpstate_idx; unsigned long val; diff --git a/drivers/cpufreq/rcpufreq_dt.rs b/drivers/cpufreq/rcpufreq_dt.rs index 94ed81644fe1..43c87d0259b6 100644 --- a/drivers/cpufreq/rcpufreq_dt.rs +++ b/drivers/cpufreq/rcpufreq_dt.rs @@ -26,9 +26,9 @@ fn find_supply_name_exact(dev: &Device, name: &str) -> Option<CString> { } /// Finds supply name for the CPU from DT. -fn find_supply_names(dev: &Device, cpu: u32) -> Option<KVec<CString>> { +fn find_supply_names(dev: &Device, cpu: cpu::CpuId) -> Option<KVec<CString>> { // Try "cpu0" for older DTs, fallback to "cpu". - let name = (cpu == 0) + let name = (cpu.as_u32() == 0) .then(|| find_supply_name_exact(dev, "cpu0")) .flatten() .or_else(|| find_supply_name_exact(dev, "cpu"))?; diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 5686369779be..9f8a3a5bed7e 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -180,6 +180,7 @@ config CRYPTO_PAES_S390 depends on PKEY select CRYPTO_ALGAPI select CRYPTO_SKCIPHER + select CRYPTO_ENGINE help This is the s390 hardware accelerated implementation of the AES cipher algorithms for use with protected key. diff --git a/drivers/crypto/axis/artpec6_crypto.c b/drivers/crypto/axis/artpec6_crypto.c index f8d50bd227a6..75ee065da1ec 100644 --- a/drivers/crypto/axis/artpec6_crypto.c +++ b/drivers/crypto/axis/artpec6_crypto.c @@ -2072,7 +2072,7 @@ static void artpec6_crypto_process_queue(struct artpec6_crypto *ac, static void artpec6_crypto_timeout(struct timer_list *t) { - struct artpec6_crypto *ac = from_timer(ac, t, timer); + struct artpec6_crypto *ac = timer_container_of(ac, t, timer); dev_info_ratelimited(artpec6_crypto_dev, "timeout\n"); diff --git a/drivers/dma-buf/st-dma-fence.c b/drivers/dma-buf/st-dma-fence.c index 261b38816226..27a36045410b 100644 --- a/drivers/dma-buf/st-dma-fence.c +++ b/drivers/dma-buf/st-dma-fence.c @@ -375,7 +375,7 @@ struct wait_timer { static void wait_timer(struct timer_list *timer) { - struct wait_timer *wt = from_timer(wt, timer, timer); + struct wait_timer *wt = timer_container_of(wt, timer, timer); dma_fence_signal(wt->f); } diff --git a/drivers/dma/imx-dma.c b/drivers/dma/imx-dma.c index b96cc0a83872..ba434657059a 100644 --- a/drivers/dma/imx-dma.c +++ b/drivers/dma/imx-dma.c @@ -337,7 +337,8 @@ static void imxdma_disable_hw(struct imxdma_channel *imxdmac) static void imxdma_watchdog(struct timer_list *t) { - struct imxdma_channel *imxdmac = from_timer(imxdmac, t, watchdog); + struct imxdma_channel *imxdmac = timer_container_of(imxdmac, t, + watchdog); struct imxdma_engine *imxdma = imxdmac->imxdma; int channel = imxdmac->channel; diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 06a813cc7641..b8fff8333aef 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -901,7 +901,8 @@ static void ioat_reboot_chan(struct ioatdma_chan *ioat_chan) void ioat_timer_event(struct timer_list *t) { - struct ioatdma_chan *ioat_chan = from_timer(ioat_chan, t, timer); + struct ioatdma_chan *ioat_chan = timer_container_of(ioat_chan, t, + timer); dma_addr_t phys_complete; u64 status; diff --git a/drivers/firewire/core-transaction.c b/drivers/firewire/core-transaction.c index 18cacb9edbbc..2bd5deb9054e 100644 --- a/drivers/firewire/core-transaction.c +++ b/drivers/firewire/core-transaction.c @@ -114,7 +114,7 @@ EXPORT_SYMBOL(fw_cancel_transaction); static void split_transaction_timeout_callback(struct timer_list *timer) { - struct fw_transaction *t = from_timer(t, timer, split_timeout_timer); + struct fw_transaction *t = timer_container_of(t, timer, split_timeout_timer); struct fw_card *card = t->card; scoped_guard(spinlock_irqsave, &card->lock) { diff --git a/drivers/firmware/cirrus/test/cs_dsp_mock_wmfw.c b/drivers/firmware/cirrus/test/cs_dsp_mock_wmfw.c index 934d40a4d709..5e1d5a810afe 100644 --- a/drivers/firmware/cirrus/test/cs_dsp_mock_wmfw.c +++ b/drivers/firmware/cirrus/test/cs_dsp_mock_wmfw.c @@ -133,10 +133,11 @@ void cs_dsp_mock_wmfw_add_info(struct cs_dsp_mock_wmfw_builder *builder, if (info_len % 4) { /* Create a padded string with length a multiple of 4 */ + size_t copy_len = info_len; info_len = round_up(info_len, 4); tmp = kunit_kzalloc(builder->test_priv->test, info_len, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(builder->test_priv->test, tmp); - memcpy(tmp, info, info_len); + memcpy(tmp, info, copy_len); info = tmp; } diff --git a/drivers/firmware/cirrus/test/cs_dsp_test_control_cache.c b/drivers/firmware/cirrus/test/cs_dsp_test_control_cache.c index 83386cc978e3..ebca3a4ab0f1 100644 --- a/drivers/firmware/cirrus/test/cs_dsp_test_control_cache.c +++ b/drivers/firmware/cirrus/test/cs_dsp_test_control_cache.c @@ -776,7 +776,6 @@ static void cs_dsp_ctl_cache_init_multiple_offsets(struct kunit *test) "dummyalg", NULL); /* Create controls identical except for offset */ - def.length_bytes = 8; def.offset_dsp_words = 0; def.shortname = "CtlA"; cs_dsp_mock_wmfw_add_coeff_desc(local->wmfw_builder, &def); diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index 2f173391b63d..939a4955e00b 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -31,7 +31,7 @@ cflags-$(CONFIG_ARM) += -DEFI_HAVE_STRLEN -DEFI_HAVE_STRNLEN \ $(DISABLE_STACKLEAK_PLUGIN) cflags-$(CONFIG_RISCV) += -fpic -DNO_ALTERNATIVE -mno-relax \ $(DISABLE_STACKLEAK_PLUGIN) -cflags-$(CONFIG_LOONGARCH) += -fpie +cflags-$(CONFIG_LOONGARCH) += -fpie $(DISABLE_STACKLEAK_PLUGIN) cflags-$(CONFIG_EFI_PARAMS_FROM_FDT) += -I$(srctree)/scripts/dtc/libfdt @@ -101,7 +101,7 @@ lib-$(CONFIG_EFI_ZBOOT) += zboot.o $(zboot-obj-y) lib-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o bitmap.o find.o -extra-y := $(lib-y) +targets := $(lib-y) lib-y := $(patsubst %.o,%.stub.o,$(lib-y)) # Even when -mbranch-protection=none is set, Clang will generate a diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 5f5c00ace96b..8cecf25996ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -322,8 +322,8 @@ bool amdgpu_fence_process(struct amdgpu_ring *ring) */ static void amdgpu_fence_fallback(struct timer_list *t) { - struct amdgpu_ring *ring = from_timer(ring, t, - fence_drv.fallback_timer); + struct amdgpu_ring *ring = timer_container_of(ring, t, + fence_drv.fallback_timer); if (amdgpu_fence_process(ring)) DRM_WARN("Fence fallback timer expired on ring %s\n", ring->name); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c index 03ed14663107..7e7d6c3865bc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c @@ -135,7 +135,8 @@ static void amdgpu_ring_mux_schedule_resubmit(struct amdgpu_ring_mux *mux) static void amdgpu_mux_resubmit_fallback(struct timer_list *t) { - struct amdgpu_ring_mux *mux = from_timer(mux, t, resubmit_timer); + struct amdgpu_ring_mux *mux = timer_container_of(mux, t, + resubmit_timer); if (!spin_trylock(&mux->lock)) { amdgpu_ring_mux_schedule_resubmit(mux); diff --git a/drivers/gpu/drm/bridge/tda998x_drv.c b/drivers/gpu/drm/bridge/tda998x_drv.c index 850909f78a7b..e636459d9185 100644 --- a/drivers/gpu/drm/bridge/tda998x_drv.c +++ b/drivers/gpu/drm/bridge/tda998x_drv.c @@ -751,7 +751,8 @@ tda998x_reset(struct tda998x_priv *priv) */ static void tda998x_edid_delay_done(struct timer_list *t) { - struct tda998x_priv *priv = from_timer(priv, t, edid_delay_timer); + struct tda998x_priv *priv = timer_container_of(priv, t, + edid_delay_timer); priv->edid_delay_active = false; wake_up(&priv->edid_delay_waitq); diff --git a/drivers/gpu/drm/drm_vblank.c b/drivers/gpu/drm/drm_vblank.c index 78958ddf8485..46f59883183d 100644 --- a/drivers/gpu/drm/drm_vblank.c +++ b/drivers/gpu/drm/drm_vblank.c @@ -487,7 +487,8 @@ out: static void vblank_disable_fn(struct timer_list *t) { - struct drm_vblank_crtc *vblank = from_timer(vblank, t, disable_timer); + struct drm_vblank_crtc *vblank = timer_container_of(vblank, t, + disable_timer); struct drm_device *dev = vblank->dev; unsigned int pipe = vblank->pipe; unsigned long irqflags; diff --git a/drivers/gpu/drm/exynos/exynos_drm_vidi.c b/drivers/gpu/drm/exynos/exynos_drm_vidi.c index e644e2382d77..e094b8bbc0f1 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_vidi.c +++ b/drivers/gpu/drm/exynos/exynos_drm_vidi.c @@ -159,7 +159,7 @@ static const struct exynos_drm_crtc_ops vidi_crtc_ops = { static void vidi_fake_vblank_timer(struct timer_list *t) { - struct vidi_context *ctx = from_timer(ctx, t, timer); + struct vidi_context *ctx = timer_container_of(ctx, t, timer); if (drm_crtc_handle_vblank(&ctx->crtc->base)) mod_timer(&ctx->timer, diff --git a/drivers/gpu/drm/gud/gud_pipe.c b/drivers/gpu/drm/gud/gud_pipe.c index feff73cc0005..adadd526641d 100644 --- a/drivers/gpu/drm/gud/gud_pipe.c +++ b/drivers/gpu/drm/gud/gud_pipe.c @@ -234,7 +234,7 @@ struct gud_usb_bulk_context { static void gud_usb_bulk_timeout(struct timer_list *t) { - struct gud_usb_bulk_context *ctx = from_timer(ctx, t, timer); + struct gud_usb_bulk_context *ctx = timer_container_of(ctx, t, timer); usb_sg_cancel(&ctx->sgr); } diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c index 5abc5fcc2514..eb89948cc112 100644 --- a/drivers/gpu/drm/i915/gt/intel_rps.c +++ b/drivers/gpu/drm/i915/gt/intel_rps.c @@ -74,7 +74,7 @@ static void set(struct intel_uncore *uncore, i915_reg_t reg, u32 val) static void rps_timer(struct timer_list *t) { - struct intel_rps *rps = from_timer(rps, t, timer); + struct intel_rps *rps = timer_container_of(rps, t, timer); struct intel_gt *gt = rps_to_gt(rps); struct intel_engine_cs *engine; ktime_t dt, last, timestamp; diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c index 64315b714743..79741f043f03 100644 --- a/drivers/gpu/drm/i915/gt/mock_engine.c +++ b/drivers/gpu/drm/i915/gt/mock_engine.c @@ -108,7 +108,7 @@ static void advance(struct i915_request *request) static void hw_delay_complete(struct timer_list *t) { - struct mock_engine *engine = from_timer(engine, t, hw_delay); + struct mock_engine *engine = timer_container_of(engine, t, hw_delay); struct i915_request *request; unsigned long flags; diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c index 32c762eb79ed..54bc447efce0 100644 --- a/drivers/gpu/drm/i915/gt/selftest_migrate.c +++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c @@ -537,7 +537,7 @@ struct spinner_timer { static void spinner_kill(struct timer_list *timer) { - struct spinner_timer *st = from_timer(st, timer, timer); + struct spinner_timer *st = timer_container_of(st, timer, timer); igt_spinner_end(&st->spin); pr_info("%s\n", __func__); diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c index 1d4cc91c0e40..a4902ee08b6e 100644 --- a/drivers/gpu/drm/i915/i915_sw_fence.c +++ b/drivers/gpu/drm/i915/i915_sw_fence.c @@ -427,7 +427,8 @@ static void dma_i915_sw_fence_wake(struct dma_fence *dma, static void timer_i915_sw_fence_wake(struct timer_list *t) { - struct i915_sw_dma_fence_cb_timer *cb = from_timer(cb, t, timer); + struct i915_sw_dma_fence_cb_timer *cb = timer_container_of(cb, t, + timer); struct i915_sw_fence *fence; fence = xchg(&cb->base.fence, NULL); diff --git a/drivers/gpu/drm/i915/intel_wakeref.c b/drivers/gpu/drm/i915/intel_wakeref.c index 07e81be4d392..51561b190b93 100644 --- a/drivers/gpu/drm/i915/intel_wakeref.c +++ b/drivers/gpu/drm/i915/intel_wakeref.c @@ -135,7 +135,7 @@ int intel_wakeref_wait_for_idle(struct intel_wakeref *wf) static void wakeref_auto_timeout(struct timer_list *t) { - struct intel_wakeref_auto *wf = from_timer(wf, t, timer); + struct intel_wakeref_auto *wf = timer_container_of(wf, t, timer); intel_wakeref_t wakeref; unsigned long flags; diff --git a/drivers/gpu/drm/i915/selftests/lib_sw_fence.c b/drivers/gpu/drm/i915/selftests/lib_sw_fence.c index 522ad49406ce..d79e4defb71d 100644 --- a/drivers/gpu/drm/i915/selftests/lib_sw_fence.c +++ b/drivers/gpu/drm/i915/selftests/lib_sw_fence.c @@ -55,7 +55,7 @@ void onstack_fence_fini(struct i915_sw_fence *fence) static void timed_fence_wake(struct timer_list *t) { - struct timed_fence *tf = from_timer(tf, t, timer); + struct timed_fence *tf = timer_container_of(tf, t, timer); i915_sw_fence_commit(&tf->fence); } diff --git a/drivers/gpu/drm/mediatek/mtk_dp.c b/drivers/gpu/drm/mediatek/mtk_dp.c index b2408abb9d49..58279ddaab3c 100644 --- a/drivers/gpu/drm/mediatek/mtk_dp.c +++ b/drivers/gpu/drm/mediatek/mtk_dp.c @@ -2569,7 +2569,7 @@ static const struct drm_bridge_funcs mtk_dp_bridge_funcs = { static void mtk_dp_debounce_timer(struct timer_list *t) { - struct mtk_dp *mtk_dp = from_timer(mtk_dp, t, debounce_timer); + struct mtk_dp *mtk_dp = timer_container_of(mtk_dp, t, debounce_timer); mtk_dp->need_debounce = true; } diff --git a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c index 36f72c43eae8..b5f9d40687d5 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c @@ -79,7 +79,8 @@ static struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) static void a5xx_preempt_timer(struct timer_list *t) { - struct a5xx_gpu *a5xx_gpu = from_timer(a5xx_gpu, t, preempt_timer); + struct a5xx_gpu *a5xx_gpu = timer_container_of(a5xx_gpu, t, + preempt_timer); struct msm_gpu *gpu = &a5xx_gpu->base.base; struct drm_device *dev = gpu->dev; diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c index 9b5e27d2373c..3b17fd2dba89 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c @@ -87,7 +87,8 @@ static struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) static void a6xx_preempt_timer(struct timer_list *t) { - struct a6xx_gpu *a6xx_gpu = from_timer(a6xx_gpu, t, preempt_timer); + struct a6xx_gpu *a6xx_gpu = timer_container_of(a6xx_gpu, t, + preempt_timer); struct msm_gpu *gpu = &a6xx_gpu->base.base; struct drm_device *dev = gpu->dev; diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c index 7020098360e4..c0ed110a7d30 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c @@ -2693,8 +2693,8 @@ static int dpu_encoder_setup_display(struct dpu_encoder_virt *dpu_enc, static void dpu_encoder_frame_done_timeout(struct timer_list *t) { - struct dpu_encoder_virt *dpu_enc = from_timer(dpu_enc, t, - frame_done_timer); + struct dpu_encoder_virt *dpu_enc = timer_container_of(dpu_enc, t, + frame_done_timer); struct drm_encoder *drm_enc = &dpu_enc->base; u32 event; diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index c380d9d9f5af..197871fdf508 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -521,7 +521,7 @@ static bool made_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring) static void hangcheck_handler(struct timer_list *t) { - struct msm_gpu *gpu = from_timer(gpu, t, hangcheck_timer); + struct msm_gpu *gpu = timer_container_of(gpu, t, hangcheck_timer); struct drm_device *dev = gpu->dev; struct msm_ringbuffer *ring = gpu->funcs->active_ring(gpu); uint32_t fence = ring->memptrs->fence; diff --git a/drivers/gpu/drm/ttm/tests/ttm_bo_test.c b/drivers/gpu/drm/ttm/tests/ttm_bo_test.c index 560d12e50e9e..6c77550c51af 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_bo_test.c +++ b/drivers/gpu/drm/ttm/tests/ttm_bo_test.c @@ -175,7 +175,7 @@ struct signal_timer { static void signal_for_ttm_bo_reserve(struct timer_list *t) { - struct signal_timer *s_timer = from_timer(s_timer, t, timer); + struct signal_timer *s_timer = timer_container_of(s_timer, t, timer); struct task_struct *task = s_timer->ctx->task; do_send_sig_info(SIGTERM, SEND_SIG_PRIV, task, PIDTYPE_PID); diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c index 7125773889f1..4aaa587be3a5 100644 --- a/drivers/gpu/drm/vc4/vc4_bo.c +++ b/drivers/gpu/drm/vc4/vc4_bo.c @@ -679,7 +679,7 @@ void vc4_bo_dec_usecnt(struct vc4_bo *bo) static void vc4_bo_cache_time_timer(struct timer_list *t) { - struct vc4_dev *vc4 = from_timer(vc4, t, bo_cache.time_timer); + struct vc4_dev *vc4 = timer_container_of(vc4, t, bo_cache.time_timer); schedule_work(&vc4->bo_cache.time_work); } diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c index 8125f87edc60..255e5817618e 100644 --- a/drivers/gpu/drm/vc4/vc4_gem.c +++ b/drivers/gpu/drm/vc4/vc4_gem.c @@ -326,7 +326,7 @@ vc4_reset_work(struct work_struct *work) static void vc4_hangcheck_elapsed(struct timer_list *t) { - struct vc4_dev *vc4 = from_timer(vc4, t, hangcheck.timer); + struct vc4_dev *vc4 = timer_container_of(vc4, t, hangcheck.timer); struct drm_device *dev = &vc4->base; uint32_t ct0ca, ct1ca; unsigned long irqflags; diff --git a/drivers/gpu/drm/vgem/vgem_fence.c b/drivers/gpu/drm/vgem/vgem_fence.c index b611c7c8ca2d..fd76730fd38c 100644 --- a/drivers/gpu/drm/vgem/vgem_fence.c +++ b/drivers/gpu/drm/vgem/vgem_fence.c @@ -61,7 +61,7 @@ static const struct dma_fence_ops vgem_fence_ops = { static void vgem_fence_timeout(struct timer_list *t) { - struct vgem_fence *fence = from_timer(fence, t, timer); + struct vgem_fence *fence = timer_container_of(fence, t, timer); dma_fence_signal(&fence->base); } diff --git a/drivers/greybus/operation.c b/drivers/greybus/operation.c index f6beeebf974c..54ccc434a1f7 100644 --- a/drivers/greybus/operation.c +++ b/drivers/greybus/operation.c @@ -295,7 +295,8 @@ static void gb_operation_work(struct work_struct *work) static void gb_operation_timeout(struct timer_list *t) { - struct gb_operation *operation = from_timer(operation, t, timer); + struct gb_operation *operation = timer_container_of(operation, t, + timer); if (gb_operation_result_set(operation, -ETIMEDOUT)) { /* diff --git a/drivers/hid/hid-apple.c b/drivers/hid/hid-apple.c index ed34f5cd5a91..0639b1f43d88 100644 --- a/drivers/hid/hid-apple.c +++ b/drivers/hid/hid-apple.c @@ -614,7 +614,7 @@ static int apple_fetch_battery(struct hid_device *hdev) static void apple_battery_timer_tick(struct timer_list *t) { - struct apple_sc *asc = from_timer(asc, t, battery_timer); + struct apple_sc *asc = timer_container_of(asc, t, battery_timer); struct hid_device *hdev = asc->hdev; if (apple_fetch_battery(hdev) == 0) { diff --git a/drivers/hid/hid-appleir.c b/drivers/hid/hid-appleir.c index bb7db9ae41c2..5e8ced7bc05a 100644 --- a/drivers/hid/hid-appleir.c +++ b/drivers/hid/hid-appleir.c @@ -167,7 +167,7 @@ static void battery_flat(struct appleir *appleir) static void key_up_tick(struct timer_list *t) { - struct appleir *appleir = from_timer(appleir, t, key_up_timer); + struct appleir *appleir = timer_container_of(appleir, t, key_up_timer); struct hid_device *hid = appleir->hid; unsigned long flags; diff --git a/drivers/hid/hid-appletb-kbd.c b/drivers/hid/hid-appletb-kbd.c index ef51b2c06872..6f251b284018 100644 --- a/drivers/hid/hid-appletb-kbd.c +++ b/drivers/hid/hid-appletb-kbd.c @@ -166,7 +166,7 @@ static int appletb_tb_key_to_slot(unsigned int code) static void appletb_inactivity_timer(struct timer_list *t) { - struct appletb_kbd *kbd = from_timer(kbd, t, inactivity_timer); + struct appletb_kbd *kbd = timer_container_of(kbd, t, inactivity_timer); if (kbd->backlight_dev && appletb_tb_autodim) { if (!kbd->has_dimmed) { diff --git a/drivers/hid/hid-letsketch.c b/drivers/hid/hid-letsketch.c index 8602c63ed9c6..11e21f988723 100644 --- a/drivers/hid/hid-letsketch.c +++ b/drivers/hid/hid-letsketch.c @@ -155,7 +155,8 @@ static int letsketch_setup_input_tablet_pad(struct letsketch_data *data) static void letsketch_inrange_timeout(struct timer_list *t) { - struct letsketch_data *data = from_timer(data, t, inrange_timer); + struct letsketch_data *data = timer_container_of(data, t, + inrange_timer); struct input_dev *input = data->input_tablet; input_report_key(input, BTN_TOOL_PEN, 0); diff --git a/drivers/hid/hid-magicmouse.c b/drivers/hid/hid-magicmouse.c index d4d91e49bbe8..36f034ac605d 100644 --- a/drivers/hid/hid-magicmouse.c +++ b/drivers/hid/hid-magicmouse.c @@ -822,7 +822,7 @@ static int magicmouse_fetch_battery(struct hid_device *hdev) static void magicmouse_battery_timer_tick(struct timer_list *t) { - struct magicmouse_sc *msc = from_timer(msc, t, battery_timer); + struct magicmouse_sc *msc = timer_container_of(msc, t, battery_timer); struct hid_device *hdev = msc->hdev; if (magicmouse_fetch_battery(hdev) == 0) { diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c index ded0fef7d8c7..b41001e02da7 100644 --- a/drivers/hid/hid-multitouch.c +++ b/drivers/hid/hid-multitouch.c @@ -1745,7 +1745,7 @@ static void mt_release_contacts(struct hid_device *hid) static void mt_expired_timeout(struct timer_list *t) { - struct mt_device *td = from_timer(td, t, release_timer); + struct mt_device *td = timer_container_of(td, t, release_timer); struct hid_device *hdev = td->hdev; /* diff --git a/drivers/hid/hid-prodikeys.c b/drivers/hid/hid-prodikeys.c index c6b922c2adba..74bddb2c3e82 100644 --- a/drivers/hid/hid-prodikeys.c +++ b/drivers/hid/hid-prodikeys.c @@ -227,7 +227,7 @@ drop_note: static void pcmidi_sustained_note_release(struct timer_list *t) { - struct pcmidi_sustain *pms = from_timer(pms, t, timer); + struct pcmidi_sustain *pms = timer_container_of(pms, t, timer); pcmidi_send_note(pms->pm, pms->status, pms->note, pms->velocity); pms->in_use = 0; diff --git a/drivers/hid/hid-sony.c b/drivers/hid/hid-sony.c index a2be652b7bbd..b966e4044238 100644 --- a/drivers/hid/hid-sony.c +++ b/drivers/hid/hid-sony.c @@ -545,7 +545,7 @@ static void ghl_magic_poke_cb(struct urb *urb) static void ghl_magic_poke(struct timer_list *t) { int ret; - struct sony_sc *sc = from_timer(sc, t, ghl_poke_timer); + struct sony_sc *sc = timer_container_of(sc, t, ghl_poke_timer); ret = usb_submit_urb(sc->ghl_urb, GFP_ATOMIC); if (ret < 0) diff --git a/drivers/hid/hid-uclogic-core.c b/drivers/hid/hid-uclogic-core.c index 61a4019ddc74..af98398d9247 100644 --- a/drivers/hid/hid-uclogic-core.c +++ b/drivers/hid/hid-uclogic-core.c @@ -32,8 +32,8 @@ */ static void uclogic_inrange_timeout(struct timer_list *t) { - struct uclogic_drvdata *drvdata = from_timer(drvdata, t, - inrange_timer); + struct uclogic_drvdata *drvdata = timer_container_of(drvdata, t, + inrange_timer); struct input_dev *input = drvdata->pen_input; if (input == NULL) diff --git a/drivers/hid/hid-wiimote-core.c b/drivers/hid/hid-wiimote-core.c index 8080083121d3..5b5fc460a4c5 100644 --- a/drivers/hid/hid-wiimote-core.c +++ b/drivers/hid/hid-wiimote-core.c @@ -1240,7 +1240,7 @@ static void wiimote_schedule(struct wiimote_data *wdata) static void wiimote_init_timeout(struct timer_list *t) { - struct wiimote_data *wdata = from_timer(wdata, t, timer); + struct wiimote_data *wdata = timer_container_of(wdata, t, timer); wiimote_schedule(wdata); } diff --git a/drivers/hid/usbhid/hid-core.c b/drivers/hid/usbhid/hid-core.c index d4cbecc668ec..aac0051a2cf6 100644 --- a/drivers/hid/usbhid/hid-core.c +++ b/drivers/hid/usbhid/hid-core.c @@ -106,7 +106,7 @@ static int hid_start_in(struct hid_device *hid) /* I/O retry timer routine */ static void hid_retry_timeout(struct timer_list *t) { - struct usbhid_device *usbhid = from_timer(usbhid, t, io_retry); + struct usbhid_device *usbhid = timer_container_of(usbhid, t, io_retry); struct hid_device *hid = usbhid->hid; dev_dbg(&usbhid->intf->dev, "retrying intr urb\n"); diff --git a/drivers/hid/wacom_wac.c b/drivers/hid/wacom_wac.c index 5107a676e24f..955b39d22524 100644 --- a/drivers/hid/wacom_wac.c +++ b/drivers/hid/wacom_wac.c @@ -63,7 +63,7 @@ static void wacom_force_proxout(struct wacom_wac *wacom_wac) void wacom_idleprox_timeout(struct timer_list *list) { - struct wacom *wacom = from_timer(wacom, list, idleprox_timer); + struct wacom *wacom = timer_container_of(wacom, list, idleprox_timer); struct wacom_wac *wacom_wac = &wacom->wacom_wac; if (!wacom_wac->hid_data.sense_state) { diff --git a/drivers/hsi/clients/ssi_protocol.c b/drivers/hsi/clients/ssi_protocol.c index cbc5e72857de..d10a01f3eb9e 100644 --- a/drivers/hsi/clients/ssi_protocol.c +++ b/drivers/hsi/clients/ssi_protocol.c @@ -453,7 +453,7 @@ static void ssip_error(struct hsi_client *cl) static void ssip_keep_alive(struct timer_list *t) { - struct ssi_protocol *ssi = from_timer(ssi, t, keep_alive); + struct ssi_protocol *ssi = timer_container_of(ssi, t, keep_alive); struct hsi_client *cl = ssi->cl; dev_dbg(&cl->device, "Keep alive kick in: m(%d) r(%d) s(%d)\n", @@ -480,7 +480,7 @@ static void ssip_keep_alive(struct timer_list *t) static void ssip_rx_wd(struct timer_list *t) { - struct ssi_protocol *ssi = from_timer(ssi, t, rx_wd); + struct ssi_protocol *ssi = timer_container_of(ssi, t, rx_wd); struct hsi_client *cl = ssi->cl; dev_err(&cl->device, "Watchdog triggered\n"); @@ -489,7 +489,7 @@ static void ssip_rx_wd(struct timer_list *t) static void ssip_tx_wd(struct timer_list *t) { - struct ssi_protocol *ssi = from_timer(ssi, t, tx_wd); + struct ssi_protocol *ssi = timer_container_of(ssi, t, tx_wd); struct hsi_client *cl = ssi->cl; dev_err(&cl->device, "Watchdog triggered\n"); diff --git a/drivers/hwmon/npcm750-pwm-fan.c b/drivers/hwmon/npcm750-pwm-fan.c index db3b551828eb..802c73def428 100644 --- a/drivers/hwmon/npcm750-pwm-fan.c +++ b/drivers/hwmon/npcm750-pwm-fan.c @@ -331,7 +331,7 @@ static void npcm7xx_fan_polling(struct timer_list *t) struct npcm7xx_pwm_fan_data *data; int i; - data = from_timer(data, t, fan_timer); + data = timer_container_of(data, t, fan_timer); /* * Polling two module per one round, diff --git a/drivers/hwmon/pwm-fan.c b/drivers/hwmon/pwm-fan.c index 2df294793f6e..d0fe53451bdf 100644 --- a/drivers/hwmon/pwm-fan.c +++ b/drivers/hwmon/pwm-fan.c @@ -78,7 +78,7 @@ static irqreturn_t pulse_handler(int irq, void *dev_id) static void sample_timer(struct timer_list *t) { - struct pwm_fan_ctx *ctx = from_timer(ctx, t, rpm_timer); + struct pwm_fan_ctx *ctx = timer_container_of(ctx, t, rpm_timer); unsigned int delta = ktime_ms_delta(ktime_get(), ctx->sample_start); int i; diff --git a/drivers/i2c/busses/i2c-img-scb.c b/drivers/i2c/busses/i2c-img-scb.c index 5d8b82ae6fd6..3278707bb885 100644 --- a/drivers/i2c/busses/i2c-img-scb.c +++ b/drivers/i2c/busses/i2c-img-scb.c @@ -831,7 +831,7 @@ next_atomic_cmd: */ static void img_i2c_check_timer(struct timer_list *t) { - struct img_i2c *i2c = from_timer(i2c, t, check_timer); + struct img_i2c *i2c = timer_container_of(i2c, t, check_timer); unsigned long flags; unsigned int line_status; diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c index 8ccb483204fa..73747d20df85 100644 --- a/drivers/idle/intel_idle.c +++ b/drivers/idle/intel_idle.c @@ -152,8 +152,8 @@ static __always_inline int __intel_idle(struct cpuidle_device *dev, int index, bool irqoff) { struct cpuidle_state *state = &drv->states[index]; - unsigned long eax = flg2MWAIT(state->flags); - unsigned long ecx = 1*irqoff; /* break on interrupt flag */ + unsigned int eax = flg2MWAIT(state->flags); + unsigned int ecx = 1*irqoff; /* break on interrupt flag */ mwait_idle_with_hints(eax, ecx); @@ -226,9 +226,9 @@ static __cpuidle int intel_idle_xstate(struct cpuidle_device *dev, static __cpuidle int intel_idle_s2idle(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { - unsigned long ecx = 1; /* break on interrupt flag */ struct cpuidle_state *state = &drv->states[index]; - unsigned long eax = flg2MWAIT(state->flags); + unsigned int eax = flg2MWAIT(state->flags); + unsigned int ecx = 1; /* break on interrupt flag */ if (state->flags & CPUIDLE_FLAG_INIT_XSTATE) fpu_idle_fpregs(); @@ -2507,6 +2507,8 @@ static int __init intel_idle_init(void) pr_debug("Local APIC timer is reliable in %s\n", boot_cpu_has(X86_FEATURE_ARAT) ? "all C-states" : "C1"); + arch_cpu_rescan_dead_smt_siblings(); + return 0; hp_setup_fail: @@ -2518,7 +2520,7 @@ init_driver_fail: return retval; } -device_initcall(intel_idle_init); +subsys_initcall_sync(intel_idle_init); /* * We are not really modular, but we used to support that. Meaning we also diff --git a/drivers/iio/common/ssp_sensors/ssp_dev.c b/drivers/iio/common/ssp_sensors/ssp_dev.c index 7c488672936f..1e167dc673ca 100644 --- a/drivers/iio/common/ssp_sensors/ssp_dev.c +++ b/drivers/iio/common/ssp_sensors/ssp_dev.c @@ -167,7 +167,7 @@ static void ssp_wdt_work_func(struct work_struct *work) static void ssp_wdt_timer_func(struct timer_list *t) { - struct ssp_data *data = from_timer(data, t, wdt_timer); + struct ssp_data *data = timer_container_of(data, t, wdt_timer); switch (data->fw_dl_state) { case SSP_FW_DL_STATE_FAIL: diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c index e02721a9e288..b3b45c49077d 100644 --- a/drivers/infiniband/hw/cxgb4/cm.c +++ b/drivers/infiniband/hw/cxgb4/cm.c @@ -4327,7 +4327,7 @@ static DECLARE_WORK(skb_work, process_work); static void ep_timeout(struct timer_list *t) { - struct c4iw_ep *ep = from_timer(ep, t, timer); + struct c4iw_ep *ep = timer_container_of(ep, t, timer); int kickit = 0; spin_lock(&timeout_lock); diff --git a/drivers/infiniband/hw/hfi1/aspm.c b/drivers/infiniband/hw/hfi1/aspm.c index 9b508eaf441d..79990d09522b 100644 --- a/drivers/infiniband/hw/hfi1/aspm.c +++ b/drivers/infiniband/hw/hfi1/aspm.c @@ -169,7 +169,7 @@ unlock: /* Timer function for re-enabling ASPM in the absence of interrupt activity */ static void aspm_ctx_timer_function(struct timer_list *t) { - struct hfi1_ctxtdata *rcd = from_timer(rcd, t, aspm_timer); + struct hfi1_ctxtdata *rcd = timer_container_of(rcd, t, aspm_timer); unsigned long flags; spin_lock_irqsave(&rcd->aspm_lock, flags); diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index e908f529335d..0781ab756d44 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -5548,7 +5548,7 @@ static void handle_cce_err(struct hfi1_devdata *dd, u32 unused, u64 reg) #define RCVERR_CHECK_TIME 10 static void update_rcverr_timer(struct timer_list *t) { - struct hfi1_devdata *dd = from_timer(dd, t, rcverr_timer); + struct hfi1_devdata *dd = timer_container_of(dd, t, rcverr_timer); struct hfi1_pportdata *ppd = dd->pport; u32 cur_ovfl_cnt = read_dev_cntr(dd, C_RCV_OVF, CNTR_INVALID_VL); @@ -12587,7 +12587,7 @@ static void do_update_synth_timer(struct work_struct *work) static void update_synth_timer(struct timer_list *t) { - struct hfi1_devdata *dd = from_timer(dd, t, synth_stats_timer); + struct hfi1_devdata *dd = timer_container_of(dd, t, synth_stats_timer); queue_work(dd->update_cntr_wq, &dd->update_cntr_work); mod_timer(&dd->synth_stats_timer, jiffies + HZ * SYNTH_CNT_TIME); diff --git a/drivers/infiniband/hw/hfi1/driver.c b/drivers/infiniband/hw/hfi1/driver.c index 3da90f2eb8e7..06487e20f723 100644 --- a/drivers/infiniband/hw/hfi1/driver.c +++ b/drivers/infiniband/hw/hfi1/driver.c @@ -1315,7 +1315,8 @@ void shutdown_led_override(struct hfi1_pportdata *ppd) static void run_led_override(struct timer_list *t) { - struct hfi1_pportdata *ppd = from_timer(ppd, t, led_override_timer); + struct hfi1_pportdata *ppd = timer_container_of(ppd, t, + led_override_timer); struct hfi1_devdata *dd = ppd->dd; unsigned long timeout; int phase_idx; diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c index b39f63ce6dfc..961fa07116f0 100644 --- a/drivers/infiniband/hw/hfi1/mad.c +++ b/drivers/infiniband/hw/hfi1/mad.c @@ -369,7 +369,7 @@ static void send_trap(struct hfi1_ibport *ibp, struct trap_node *trap) void hfi1_handle_trap_timer(struct timer_list *t) { - struct hfi1_ibport *ibp = from_timer(ibp, t, rvp.trap_timer); + struct hfi1_ibport *ibp = timer_container_of(ibp, t, rvp.trap_timer); struct trap_node *trap = NULL; unsigned long flags; int i; diff --git a/drivers/infiniband/hw/hfi1/sdma.c b/drivers/infiniband/hw/hfi1/sdma.c index 16a749d16ee9..719b7c34e238 100644 --- a/drivers/infiniband/hw/hfi1/sdma.c +++ b/drivers/infiniband/hw/hfi1/sdma.c @@ -467,7 +467,8 @@ static void sdma_err_progress_check_schedule(struct sdma_engine *sde) static void sdma_err_progress_check(struct timer_list *t) { unsigned index; - struct sdma_engine *sde = from_timer(sde, t, err_progress_check_timer); + struct sdma_engine *sde = timer_container_of(sde, t, + err_progress_check_timer); dd_dev_err(sde->dd, "SDE progress check event\n"); for (index = 0; index < sde->dd->num_sdma; index++) { diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c index 78bf4a48c035..eafd2f157e32 100644 --- a/drivers/infiniband/hw/hfi1/tid_rdma.c +++ b/drivers/infiniband/hw/hfi1/tid_rdma.c @@ -3981,7 +3981,7 @@ void hfi1_del_tid_reap_timer(struct rvt_qp *qp) static void hfi1_tid_timeout(struct timer_list *t) { - struct hfi1_qp_priv *qpriv = from_timer(qpriv, t, s_tid_timer); + struct hfi1_qp_priv *qpriv = timer_container_of(qpriv, t, s_tid_timer); struct rvt_qp *qp = qpriv->owner; struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device); unsigned long flags; @@ -4797,7 +4797,8 @@ void hfi1_del_tid_retry_timer(struct rvt_qp *qp) static void hfi1_tid_retry_timeout(struct timer_list *t) { - struct hfi1_qp_priv *priv = from_timer(priv, t, s_tid_retry_timer); + struct hfi1_qp_priv *priv = timer_container_of(priv, t, + s_tid_retry_timer); struct rvt_qp *qp = priv->owner; struct rvt_swqe *wqe; unsigned long flags; diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index 49e0f79b950c..3cbbfccdd8cd 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -554,7 +554,7 @@ void hfi1_16B_rcv(struct hfi1_packet *packet) */ static void mem_timer(struct timer_list *t) { - struct hfi1_ibdev *dev = from_timer(dev, t, mem_timer); + struct hfi1_ibdev *dev = timer_container_of(dev, t, mem_timer); struct list_head *list = &dev->memwait; struct rvt_qp *qp = NULL; struct iowait *wait; diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c index 23207f13ac1b..c6a0a661d6e7 100644 --- a/drivers/infiniband/hw/irdma/cm.c +++ b/drivers/infiniband/hw/irdma/cm.c @@ -1263,7 +1263,8 @@ static void irdma_cm_timer_tick(struct timer_list *t) struct irdma_timer_entry *send_entry, *close_entry; struct list_head *list_core_temp; struct list_head *list_node; - struct irdma_cm_core *cm_core = from_timer(cm_core, t, tcp_timer); + struct irdma_cm_core *cm_core = timer_container_of(cm_core, t, + tcp_timer); struct irdma_sc_vsi *vsi; u32 settimer = 0; unsigned long timetosend; diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c index d66b4f7a84ec..b510ef747399 100644 --- a/drivers/infiniband/hw/irdma/utils.c +++ b/drivers/infiniband/hw/irdma/utils.c @@ -930,7 +930,7 @@ void irdma_terminate_done(struct irdma_sc_qp *qp, int timeout_occurred) static void irdma_terminate_timeout(struct timer_list *t) { - struct irdma_qp *iwqp = from_timer(iwqp, t, terminate_timer); + struct irdma_qp *iwqp = timer_container_of(iwqp, t, terminate_timer); struct irdma_sc_qp *qp = &iwqp->sc_qp; irdma_terminate_done(qp, 1); @@ -1537,7 +1537,7 @@ int irdma_puda_get_tcpip_info(struct irdma_puda_cmpl_info *info, static void irdma_hw_stats_timeout(struct timer_list *t) { struct irdma_vsi_pestat *pf_devstat = - from_timer(pf_devstat, t, stats_timer); + timer_container_of(pf_devstat, t, stats_timer); struct irdma_sc_vsi *sc_vsi = pf_devstat->vsi; if (sc_vsi->dev->hw_attrs.uk_attrs.hw_rev >= IRDMA_GEN_2) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 6dd813bac5b2..57f9bc2a4a3a 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -838,7 +838,7 @@ static void mlx5_mkey_cache_debugfs_init(struct mlx5_ib_dev *dev) static void delay_time_func(struct timer_list *t) { - struct mlx5_ib_dev *dev = from_timer(dev, t, delay_timer); + struct mlx5_ib_dev *dev = timer_container_of(dev, t, delay_timer); WRITE_ONCE(dev->fill_delay, 0); } diff --git a/drivers/infiniband/hw/mthca/mthca_catas.c b/drivers/infiniband/hw/mthca/mthca_catas.c index 6eabef9aa211..f1d79968c985 100644 --- a/drivers/infiniband/hw/mthca/mthca_catas.c +++ b/drivers/infiniband/hw/mthca/mthca_catas.c @@ -132,7 +132,7 @@ static void handle_catas(struct mthca_dev *dev) static void poll_catas(struct timer_list *t) { - struct mthca_dev *dev = from_timer(dev, t, catas_err.timer); + struct mthca_dev *dev = timer_container_of(dev, t, catas_err.timer); int i; for (i = 0; i < dev->catas_err.size; ++i) diff --git a/drivers/infiniband/hw/qib/qib_driver.c b/drivers/infiniband/hw/qib/qib_driver.c index bdd724add147..91fa5e160c0d 100644 --- a/drivers/infiniband/hw/qib/qib_driver.c +++ b/drivers/infiniband/hw/qib/qib_driver.c @@ -659,8 +659,8 @@ int qib_set_lid(struct qib_pportdata *ppd, u32 lid, u8 lmc) static void qib_run_led_override(struct timer_list *t) { - struct qib_pportdata *ppd = from_timer(ppd, t, - led_override_timer); + struct qib_pportdata *ppd = timer_container_of(ppd, t, + led_override_timer); struct qib_devdata *dd = ppd->dd; int timeoff; int ph_idx; diff --git a/drivers/infiniband/hw/qib/qib_iba6120.c b/drivers/infiniband/hw/qib/qib_iba6120.c index 277769fa9745..2640d283eee6 100644 --- a/drivers/infiniband/hw/qib/qib_iba6120.c +++ b/drivers/infiniband/hw/qib/qib_iba6120.c @@ -2615,7 +2615,7 @@ static void qib_chk_6120_errormask(struct qib_devdata *dd) */ static void qib_get_6120_faststats(struct timer_list *t) { - struct qib_devdata *dd = from_timer(dd, t, stats_timer); + struct qib_devdata *dd = timer_container_of(dd, t, stats_timer); struct qib_pportdata *ppd = dd->pport; unsigned long flags; u64 traffic_wds; @@ -2905,7 +2905,7 @@ static int qib_6120_set_loopback(struct qib_pportdata *ppd, const char *what) static void pma_6120_timer(struct timer_list *t) { - struct qib_chip_specific *cs = from_timer(cs, t, pma_timer); + struct qib_chip_specific *cs = timer_container_of(cs, t, pma_timer); struct qib_pportdata *ppd = cs->ppd; struct qib_ibport *ibp = &ppd->ibport_data; unsigned long flags; diff --git a/drivers/infiniband/hw/qib/qib_iba7220.c b/drivers/infiniband/hw/qib/qib_iba7220.c index 302c0d19f57d..0b347d1129fa 100644 --- a/drivers/infiniband/hw/qib/qib_iba7220.c +++ b/drivers/infiniband/hw/qib/qib_iba7220.c @@ -1044,8 +1044,8 @@ done: static void reenable_7220_chase(struct timer_list *t) { - struct qib_chippport_specific *cpspec = from_timer(cpspec, t, - chase_timer); + struct qib_chippport_specific *cpspec = timer_container_of(cpspec, t, + chase_timer); struct qib_pportdata *ppd = &cpspec->pportdata; ppd->cpspec->chase_timer.expires = 0; @@ -3240,7 +3240,7 @@ done: */ static void qib_get_7220_faststats(struct timer_list *t) { - struct qib_devdata *dd = from_timer(dd, t, stats_timer); + struct qib_devdata *dd = timer_container_of(dd, t, stats_timer); struct qib_pportdata *ppd = dd->pport; unsigned long flags; u64 traffic_wds; diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c b/drivers/infiniband/hw/qib/qib_iba7322.c index 7b4bf06c3b38..781b6a4fb002 100644 --- a/drivers/infiniband/hw/qib/qib_iba7322.c +++ b/drivers/infiniband/hw/qib/qib_iba7322.c @@ -1721,7 +1721,8 @@ static void qib_error_tasklet(struct tasklet_struct *t) static void reenable_chase(struct timer_list *t) { - struct qib_chippport_specific *cp = from_timer(cp, t, chase_timer); + struct qib_chippport_specific *cp = timer_container_of(cp, t, + chase_timer); struct qib_pportdata *ppd = cp->ppd; ppd->cpspec->chase_timer.expires = 0; @@ -5084,7 +5085,7 @@ done: */ static void qib_get_7322_faststats(struct timer_list *t) { - struct qib_devdata *dd = from_timer(dd, t, stats_timer); + struct qib_devdata *dd = timer_container_of(dd, t, stats_timer); struct qib_pportdata *ppd; unsigned long flags; u64 traffic_wds; diff --git a/drivers/infiniband/hw/qib/qib_init.c b/drivers/infiniband/hw/qib/qib_init.c index 33c23adec101..1c45814f5646 100644 --- a/drivers/infiniband/hw/qib/qib_init.c +++ b/drivers/infiniband/hw/qib/qib_init.c @@ -493,7 +493,7 @@ static void enable_chip(struct qib_devdata *dd) static void verify_interrupt(struct timer_list *t) { - struct qib_devdata *dd = from_timer(dd, t, intrchk_timer); + struct qib_devdata *dd = timer_container_of(dd, t, intrchk_timer); u64 int_counter; if (!dd) diff --git a/drivers/infiniband/hw/qib/qib_intr.c b/drivers/infiniband/hw/qib/qib_intr.c index 85c3187d796d..93357823c6c0 100644 --- a/drivers/infiniband/hw/qib/qib_intr.c +++ b/drivers/infiniband/hw/qib/qib_intr.c @@ -172,7 +172,8 @@ skip_ibchange: void qib_clear_symerror_on_linkup(struct timer_list *t) { - struct qib_pportdata *ppd = from_timer(ppd, t, symerr_clear_timer); + struct qib_pportdata *ppd = timer_container_of(ppd, t, + symerr_clear_timer); if (ppd->lflags & QIBL_LINKACTIVE) return; diff --git a/drivers/infiniband/hw/qib/qib_mad.c b/drivers/infiniband/hw/qib/qib_mad.c index 568deb77ab4d..d99932b2ce21 100644 --- a/drivers/infiniband/hw/qib/qib_mad.c +++ b/drivers/infiniband/hw/qib/qib_mad.c @@ -2398,7 +2398,8 @@ bail: static void xmit_wait_timer_func(struct timer_list *t) { - struct qib_pportdata *ppd = from_timer(ppd, t, cong_stats.timer); + struct qib_pportdata *ppd = timer_container_of(ppd, t, + cong_stats.timer); struct qib_devdata *dd = dd_from_ppd(ppd); unsigned long flags; u8 status; diff --git a/drivers/infiniband/hw/qib/qib_sd7220.c b/drivers/infiniband/hw/qib/qib_sd7220.c index c4ee120ac7fb..40bc0a34273e 100644 --- a/drivers/infiniband/hw/qib/qib_sd7220.c +++ b/drivers/infiniband/hw/qib/qib_sd7220.c @@ -1385,7 +1385,7 @@ MODULE_PARM_DESC(relock_by_timer, "Allow relock attempt if link not up"); static void qib_run_relock(struct timer_list *t) { - struct qib_chip_specific *cs = from_timer(cs, t, relock_timer); + struct qib_chip_specific *cs = timer_container_of(cs, t, relock_timer); struct qib_devdata *dd = cs->dd; struct qib_pportdata *ppd = dd->pport; int timeoff; diff --git a/drivers/infiniband/hw/qib/qib_tx.c b/drivers/infiniband/hw/qib/qib_tx.c index 1325110237cd..397928c80f7c 100644 --- a/drivers/infiniband/hw/qib/qib_tx.c +++ b/drivers/infiniband/hw/qib/qib_tx.c @@ -548,7 +548,7 @@ void qib_hol_up(struct qib_pportdata *ppd) */ void qib_hol_event(struct timer_list *t) { - struct qib_pportdata *ppd = from_timer(ppd, t, hol_timer); + struct qib_pportdata *ppd = timer_container_of(ppd, t, hol_timer); /* If hardware error, etc, skip. */ if (!(ppd->dd->flags & QIB_INITTED)) diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c index 9832567a8bb8..bab657f93084 100644 --- a/drivers/infiniband/hw/qib/qib_verbs.c +++ b/drivers/infiniband/hw/qib/qib_verbs.c @@ -361,7 +361,7 @@ drop: */ static void mem_timer(struct timer_list *t) { - struct qib_ibdev *dev = from_timer(dev, t, mem_timer); + struct qib_ibdev *dev = timer_container_of(dev, t, mem_timer); struct list_head *list = &dev->memwait; struct rvt_qp *qp = NULL; struct qib_qp_priv *priv = NULL; diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c index 583debe4b9a2..e825e2ef7966 100644 --- a/drivers/infiniband/sw/rdmavt/qp.c +++ b/drivers/infiniband/sw/rdmavt/qp.c @@ -2585,7 +2585,7 @@ EXPORT_SYMBOL(rvt_del_timers_sync); */ static void rvt_rc_timeout(struct timer_list *t) { - struct rvt_qp *qp = from_timer(qp, t, s_timer); + struct rvt_qp *qp = timer_container_of(qp, t, s_timer); struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device); unsigned long flags; diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index d48af2180745..a5b2b62f596b 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -114,7 +114,7 @@ static enum ib_wc_opcode wr_to_wc_opcode(enum ib_wr_opcode opcode) void retransmit_timer(struct timer_list *t) { - struct rxe_qp *qp = from_timer(qp, t, retrans_timer); + struct rxe_qp *qp = timer_container_of(qp, t, retrans_timer); unsigned long flags; rxe_dbg_qp(qp, "retransmit timer fired\n"); diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 9d0392df8a92..373b03f223be 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -97,7 +97,7 @@ static void req_retry(struct rxe_qp *qp) void rnr_nak_timer(struct timer_list *t) { - struct rxe_qp *qp = from_timer(qp, t, rnr_nak_timer); + struct rxe_qp *qp = timer_container_of(qp, t, rnr_nak_timer); unsigned long flags; rxe_dbg_qp(qp, "nak timer fired\n"); diff --git a/drivers/input/ff-memless.c b/drivers/input/ff-memless.c index 009822fa61b8..91636479ee3c 100644 --- a/drivers/input/ff-memless.c +++ b/drivers/input/ff-memless.c @@ -399,7 +399,7 @@ static void ml_play_effects(struct ml_device *ml) static void ml_effect_timer(struct timer_list *t) { - struct ml_device *ml = from_timer(ml, t, timer); + struct ml_device *ml = timer_container_of(ml, t, timer); struct input_dev *dev = ml->dev; pr_debug("timer: updating effects\n"); diff --git a/drivers/input/gameport/gameport.c b/drivers/input/gameport/gameport.c index ae51f108bfae..a832bc46bc92 100644 --- a/drivers/input/gameport/gameport.c +++ b/drivers/input/gameport/gameport.c @@ -199,7 +199,8 @@ EXPORT_SYMBOL(gameport_stop_polling); static void gameport_run_poll_handler(struct timer_list *t) { - struct gameport *gameport = from_timer(gameport, t, poll_timer); + struct gameport *gameport = timer_container_of(gameport, t, + poll_timer); gameport->poll_handler(gameport); if (gameport->poll_cnt) diff --git a/drivers/input/input.c b/drivers/input/input.c index ec4346f20efd..44887e51e049 100644 --- a/drivers/input/input.c +++ b/drivers/input/input.c @@ -2249,7 +2249,7 @@ static void devm_input_device_unregister(struct device *dev, void *res) */ static void input_repeat_key(struct timer_list *t) { - struct input_dev *dev = from_timer(dev, t, timer); + struct input_dev *dev = timer_container_of(dev, t, timer); guard(spinlock_irqsave)(&dev->event_lock); diff --git a/drivers/input/joystick/db9.c b/drivers/input/joystick/db9.c index d7a253835889..d5c67a927404 100644 --- a/drivers/input/joystick/db9.c +++ b/drivers/input/joystick/db9.c @@ -351,7 +351,7 @@ static int db9_saturn(int mode, struct parport *port, struct input_dev *devs[]) static void db9_timer(struct timer_list *t) { - struct db9 *db9 = from_timer(db9, t, timer); + struct db9 *db9 = timer_container_of(db9, t, timer); struct parport *port = db9->pd->port; struct input_dev *dev = db9->dev[0]; struct input_dev *dev2 = db9->dev[1]; diff --git a/drivers/input/joystick/gamecon.c b/drivers/input/joystick/gamecon.c index 9fc629ad58b8..ae95cb3d0ae9 100644 --- a/drivers/input/joystick/gamecon.c +++ b/drivers/input/joystick/gamecon.c @@ -726,7 +726,7 @@ static void gc_psx_process_packet(struct gc *gc) static void gc_timer(struct timer_list *t) { - struct gc *gc = from_timer(gc, t, timer); + struct gc *gc = timer_container_of(gc, t, timer); /* * N64 pads - must be read first, any read confuses them for 200 us diff --git a/drivers/input/joystick/turbografx.c b/drivers/input/joystick/turbografx.c index aa3e7d471b96..5f69aef01791 100644 --- a/drivers/input/joystick/turbografx.c +++ b/drivers/input/joystick/turbografx.c @@ -72,7 +72,7 @@ static struct tgfx { static void tgfx_timer(struct timer_list *t) { - struct tgfx *tgfx = from_timer(tgfx, t, timer); + struct tgfx *tgfx = timer_container_of(tgfx, t, timer); struct input_dev *dev; int data1, data2, i; diff --git a/drivers/input/keyboard/imx_keypad.c b/drivers/input/keyboard/imx_keypad.c index 3cd47fa44efc..069c1d6376e1 100644 --- a/drivers/input/keyboard/imx_keypad.c +++ b/drivers/input/keyboard/imx_keypad.c @@ -183,7 +183,8 @@ static void imx_keypad_fire_events(struct imx_keypad *keypad, */ static void imx_keypad_check_for_events(struct timer_list *t) { - struct imx_keypad *keypad = from_timer(keypad, t, check_matrix_timer); + struct imx_keypad *keypad = timer_container_of(keypad, t, + check_matrix_timer); unsigned short matrix_volatile_state[MAX_MATRIX_KEY_COLS]; unsigned short reg_val; bool state_changed, is_zero_matrix; diff --git a/drivers/input/keyboard/locomokbd.c b/drivers/input/keyboard/locomokbd.c index c501a93a4417..58d4f2096cf9 100644 --- a/drivers/input/keyboard/locomokbd.c +++ b/drivers/input/keyboard/locomokbd.c @@ -194,7 +194,7 @@ static irqreturn_t locomokbd_interrupt(int irq, void *dev_id) */ static void locomokbd_timer_callback(struct timer_list *t) { - struct locomokbd *locomokbd = from_timer(locomokbd, t, timer); + struct locomokbd *locomokbd = timer_container_of(locomokbd, t, timer); locomokbd_scankeyboard(locomokbd); } diff --git a/drivers/input/keyboard/snvs_pwrkey.c b/drivers/input/keyboard/snvs_pwrkey.c index fe7398eeb828..954055aaf6e2 100644 --- a/drivers/input/keyboard/snvs_pwrkey.c +++ b/drivers/input/keyboard/snvs_pwrkey.c @@ -46,7 +46,8 @@ struct pwrkey_drv_data { static void imx_imx_snvs_check_for_events(struct timer_list *t) { - struct pwrkey_drv_data *pdata = from_timer(pdata, t, check_timer); + struct pwrkey_drv_data *pdata = timer_container_of(pdata, t, + check_timer); struct input_dev *input = pdata->input; u32 state; diff --git a/drivers/input/keyboard/tegra-kbc.c b/drivers/input/keyboard/tegra-kbc.c index 32a676f0de53..bc1c80a456f2 100644 --- a/drivers/input/keyboard/tegra-kbc.c +++ b/drivers/input/keyboard/tegra-kbc.c @@ -240,7 +240,7 @@ static void tegra_kbc_set_fifo_interrupt(struct tegra_kbc *kbc, bool enable) static void tegra_kbc_keypress_timer(struct timer_list *t) { - struct tegra_kbc *kbc = from_timer(kbc, t, timer); + struct tegra_kbc *kbc = timer_container_of(kbc, t, timer); u32 val; unsigned int i; diff --git a/drivers/input/misc/nxp-bbnsm-pwrkey.c b/drivers/input/misc/nxp-bbnsm-pwrkey.c index 7ba8d166d68c..0c7b8f8ef4a5 100644 --- a/drivers/input/misc/nxp-bbnsm-pwrkey.c +++ b/drivers/input/misc/nxp-bbnsm-pwrkey.c @@ -45,7 +45,7 @@ struct bbnsm_pwrkey { static void bbnsm_pwrkey_check_for_events(struct timer_list *t) { - struct bbnsm_pwrkey *bbnsm = from_timer(bbnsm, t, check_timer); + struct bbnsm_pwrkey *bbnsm = timer_container_of(bbnsm, t, check_timer); struct input_dev *input = bbnsm->input; u32 state; diff --git a/drivers/input/mouse/alps.c b/drivers/input/mouse/alps.c index 0bd7b09b0aa3..be734d65ea72 100644 --- a/drivers/input/mouse/alps.c +++ b/drivers/input/mouse/alps.c @@ -1582,7 +1582,7 @@ static psmouse_ret_t alps_handle_interleaved_ps2(struct psmouse *psmouse) static void alps_flush_packet(struct timer_list *t) { - struct alps_data *priv = from_timer(priv, t, timer); + struct alps_data *priv = timer_container_of(priv, t, timer); struct psmouse *psmouse = priv->psmouse; guard(serio_pause_rx)(psmouse->ps2dev.serio); diff --git a/drivers/input/mouse/byd.c b/drivers/input/mouse/byd.c index 4ee084e00a7c..71aa23dd7d8d 100644 --- a/drivers/input/mouse/byd.c +++ b/drivers/input/mouse/byd.c @@ -251,7 +251,7 @@ static void byd_report_input(struct psmouse *psmouse) static void byd_clear_touch(struct timer_list *t) { - struct byd_data *priv = from_timer(priv, t, timer); + struct byd_data *priv = timer_container_of(priv, t, timer); struct psmouse *psmouse = priv->psmouse; guard(serio_pause_rx)(psmouse->ps2dev.serio); diff --git a/drivers/input/touchscreen/ad7877.c b/drivers/input/touchscreen/ad7877.c index 8d8392ce7005..c9aa1847265a 100644 --- a/drivers/input/touchscreen/ad7877.c +++ b/drivers/input/touchscreen/ad7877.c @@ -375,7 +375,7 @@ static inline void ad7877_ts_event_release(struct ad7877 *ts) static void ad7877_timer(struct timer_list *t) { - struct ad7877 *ts = from_timer(ts, t, timer); + struct ad7877 *ts = timer_container_of(ts, t, timer); unsigned long flags; spin_lock_irqsave(&ts->lock, flags); diff --git a/drivers/input/touchscreen/ad7879.c b/drivers/input/touchscreen/ad7879.c index f661e199b63c..f9db5cefb25b 100644 --- a/drivers/input/touchscreen/ad7879.c +++ b/drivers/input/touchscreen/ad7879.c @@ -237,7 +237,7 @@ static void ad7879_ts_event_release(struct ad7879 *ts) static void ad7879_timer(struct timer_list *t) { - struct ad7879 *ts = from_timer(ts, t, timer); + struct ad7879 *ts = timer_container_of(ts, t, timer); ad7879_ts_event_release(ts); } diff --git a/drivers/input/touchscreen/bu21029_ts.c b/drivers/input/touchscreen/bu21029_ts.c index 3c997fba7048..64f474e67312 100644 --- a/drivers/input/touchscreen/bu21029_ts.c +++ b/drivers/input/touchscreen/bu21029_ts.c @@ -209,7 +209,8 @@ static void bu21029_touch_report(struct bu21029_ts_data *bu21029, const u8 *buf) static void bu21029_touch_release(struct timer_list *t) { - struct bu21029_ts_data *bu21029 = from_timer(bu21029, t, timer); + struct bu21029_ts_data *bu21029 = timer_container_of(bu21029, t, + timer); input_report_abs(bu21029->in_dev, ABS_PRESSURE, 0); input_report_key(bu21029->in_dev, BTN_TOUCH, 0); diff --git a/drivers/input/touchscreen/exc3000.c b/drivers/input/touchscreen/exc3000.c index 9a5977d8cad2..28da7ba55a4b 100644 --- a/drivers/input/touchscreen/exc3000.c +++ b/drivers/input/touchscreen/exc3000.c @@ -105,7 +105,7 @@ static void exc3000_report_slots(struct input_dev *input, static void exc3000_timer(struct timer_list *t) { - struct exc3000_data *data = from_timer(data, t, timer); + struct exc3000_data *data = timer_container_of(data, t, timer); input_mt_sync_frame(data->input); input_sync(data->input); diff --git a/drivers/input/touchscreen/sx8654.c b/drivers/input/touchscreen/sx8654.c index e59b8d0ed19e..5fa47a1a6fdc 100644 --- a/drivers/input/touchscreen/sx8654.c +++ b/drivers/input/touchscreen/sx8654.c @@ -116,7 +116,7 @@ static inline void sx865x_penrelease(struct sx8654 *ts) static void sx865x_penrelease_timer_handler(struct timer_list *t) { - struct sx8654 *ts = from_timer(ts, t, timer); + struct sx8654 *ts = timer_container_of(ts, t, timer); unsigned long flags; spin_lock_irqsave(&ts->lock, flags); diff --git a/drivers/input/touchscreen/tsc200x-core.c b/drivers/input/touchscreen/tsc200x-core.c index 252a93753ee5..82d7d1cf5010 100644 --- a/drivers/input/touchscreen/tsc200x-core.c +++ b/drivers/input/touchscreen/tsc200x-core.c @@ -194,7 +194,7 @@ out: static void tsc200x_penup_timer(struct timer_list *t) { - struct tsc200x *ts = from_timer(ts, t, penup_timer); + struct tsc200x *ts = timer_container_of(ts, t, penup_timer); guard(spinlock_irqsave)(&ts->lock); tsc200x_update_pen_state(ts, 0, 0, 0); diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 6c708fec48d1..ea2ef53bd4fe 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -179,7 +179,8 @@ static void fq_flush_iotlb(struct iommu_dma_cookie *cookie) static void fq_flush_timeout(struct timer_list *t) { - struct iommu_dma_cookie *cookie = from_timer(cookie, t, fq_timer); + struct iommu_dma_cookie *cookie = timer_container_of(cookie, t, + fq_timer); int cpu; atomic_set(&cookie->fq_timer_on, 0); diff --git a/drivers/isdn/hardware/mISDN/hfcpci.c b/drivers/isdn/hardware/mISDN/hfcpci.c index e3870d942699..2b05722d4dbe 100644 --- a/drivers/isdn/hardware/mISDN/hfcpci.c +++ b/drivers/isdn/hardware/mISDN/hfcpci.c @@ -291,7 +291,7 @@ reset_hfcpci(struct hfc_pci *hc) static void hfcpci_Timer(struct timer_list *t) { - struct hfc_pci *hc = from_timer(hc, t, hw.timer); + struct hfc_pci *hc = timer_container_of(hc, t, hw.timer); hc->hw.timer.expires = jiffies + 75; /* WD RESET */ /* diff --git a/drivers/isdn/hardware/mISDN/mISDNipac.c b/drivers/isdn/hardware/mISDN/mISDNipac.c index 165a63994f04..a34ea6058960 100644 --- a/drivers/isdn/hardware/mISDN/mISDNipac.c +++ b/drivers/isdn/hardware/mISDN/mISDNipac.c @@ -713,7 +713,7 @@ isac_release(struct isac_hw *isac) static void dbusy_timer_handler(struct timer_list *t) { - struct isac_hw *isac = from_timer(isac, t, dch.timer); + struct isac_hw *isac = timer_container_of(isac, t, dch.timer); int rbch, star; u_long flags; diff --git a/drivers/isdn/hardware/mISDN/mISDNisar.c b/drivers/isdn/hardware/mISDN/mISDNisar.c index e48f89cbecf8..dace91ba412b 100644 --- a/drivers/isdn/hardware/mISDN/mISDNisar.c +++ b/drivers/isdn/hardware/mISDN/mISDNisar.c @@ -1135,7 +1135,7 @@ EXPORT_SYMBOL(mISDNisar_irq); static void ftimer_handler(struct timer_list *t) { - struct isar_ch *ch = from_timer(ch, t, ftimer); + struct isar_ch *ch = timer_container_of(ch, t, ftimer); pr_debug("%s: ftimer flags %lx\n", ch->is->name, ch->bch.Flags); test_and_clear_bit(FLG_FTI_RUN, &ch->bch.Flags); diff --git a/drivers/isdn/hardware/mISDN/w6692.c b/drivers/isdn/hardware/mISDN/w6692.c index da933f4bdf4e..168fc345dcdc 100644 --- a/drivers/isdn/hardware/mISDN/w6692.c +++ b/drivers/isdn/hardware/mISDN/w6692.c @@ -802,7 +802,7 @@ w6692_irq(int intno, void *dev_id) static void dbusy_timer_handler(struct timer_list *t) { - struct dchannel *dch = from_timer(dch, t, timer); + struct dchannel *dch = timer_container_of(dch, t, timer); struct w6692_hw *card = dch->hw; int rbch, star; u_long flags; diff --git a/drivers/isdn/mISDN/dsp_tones.c b/drivers/isdn/mISDN/dsp_tones.c index 456b313bfa76..fa7813ae8d97 100644 --- a/drivers/isdn/mISDN/dsp_tones.c +++ b/drivers/isdn/mISDN/dsp_tones.c @@ -459,7 +459,7 @@ dsp_tone_hw_message(struct dsp *dsp, u8 *sample, int len) void dsp_tone_timeout(struct timer_list *t) { - struct dsp *dsp = from_timer(dsp, t, tone.tl); + struct dsp *dsp = timer_container_of(dsp, t, tone.tl); struct dsp_tone *tone = &dsp->tone; struct pattern *pat = (struct pattern *)tone->pattern; int index = tone->index; diff --git a/drivers/isdn/mISDN/fsm.c b/drivers/isdn/mISDN/fsm.c index 5ed0a6110687..825b686496d2 100644 --- a/drivers/isdn/mISDN/fsm.c +++ b/drivers/isdn/mISDN/fsm.c @@ -95,7 +95,7 @@ EXPORT_SYMBOL(mISDN_FsmChangeState); static void FsmExpireTimer(struct timer_list *t) { - struct FsmTimer *ft = from_timer(ft, t, tl); + struct FsmTimer *ft = timer_container_of(ft, t, tl); #if FSM_TIMER_DEBUG if (ft->fi->debug) ft->fi->printdebug(ft->fi, "FsmExpireTimer %lx", (long) ft); diff --git a/drivers/isdn/mISDN/l1oip_core.c b/drivers/isdn/mISDN/l1oip_core.c index a5ad88a960d0..f732f6614d37 100644 --- a/drivers/isdn/mISDN/l1oip_core.c +++ b/drivers/isdn/mISDN/l1oip_core.c @@ -822,7 +822,7 @@ l1oip_send_bh(struct work_struct *work) static void l1oip_keepalive(struct timer_list *t) { - struct l1oip *hc = from_timer(hc, t, keep_tl); + struct l1oip *hc = timer_container_of(hc, t, keep_tl); schedule_work(&hc->workq); } @@ -830,8 +830,8 @@ l1oip_keepalive(struct timer_list *t) static void l1oip_timeout(struct timer_list *t) { - struct l1oip *hc = from_timer(hc, t, - timeout_tl); + struct l1oip *hc = timer_container_of(hc, t, + timeout_tl); struct dchannel *dch = hc->chan[hc->d_idx].dch; if (debug & DEBUG_L1OIP_MSG) diff --git a/drivers/isdn/mISDN/timerdev.c b/drivers/isdn/mISDN/timerdev.c index 7cfa8c61dba0..df98144a9539 100644 --- a/drivers/isdn/mISDN/timerdev.c +++ b/drivers/isdn/mISDN/timerdev.c @@ -155,7 +155,7 @@ mISDN_poll(struct file *filep, poll_table *wait) static void dev_expire_timer(struct timer_list *t) { - struct mISDNtimer *timer = from_timer(timer, t, tl); + struct mISDNtimer *timer = timer_container_of(timer, t, tl); u_long flags; spin_lock_irqsave(&timer->dev->lock, flags); diff --git a/drivers/leds/flash/leds-rt8515.c b/drivers/leds/flash/leds-rt8515.c index 32ba397a33d2..6af0d2c7fc56 100644 --- a/drivers/leds/flash/leds-rt8515.c +++ b/drivers/leds/flash/leds-rt8515.c @@ -165,7 +165,7 @@ static const struct led_flash_ops rt8515_flash_ops = { static void rt8515_powerdown_timer(struct timer_list *t) { - struct rt8515 *rt = from_timer(rt, t, powerdown_timer); + struct rt8515 *rt = timer_container_of(rt, t, powerdown_timer); /* Turn the LED off */ rt8515_gpio_led_off(rt); diff --git a/drivers/leds/flash/leds-sgm3140.c b/drivers/leds/flash/leds-sgm3140.c index 48fb8a9ec703..3e83200675f2 100644 --- a/drivers/leds/flash/leds-sgm3140.c +++ b/drivers/leds/flash/leds-sgm3140.c @@ -135,7 +135,7 @@ static int sgm3140_brightness_set(struct led_classdev *led_cdev, static void sgm3140_powerdown_timer(struct timer_list *t) { - struct sgm3140 *priv = from_timer(priv, t, powerdown_timer); + struct sgm3140 *priv = timer_container_of(priv, t, powerdown_timer); gpiod_set_value(priv->enable_gpio, 0); gpiod_set_value(priv->flash_gpio, 0); diff --git a/drivers/leds/led-core.c b/drivers/leds/led-core.c index 1a59a4f38479..59473f286b31 100644 --- a/drivers/leds/led-core.c +++ b/drivers/leds/led-core.c @@ -64,7 +64,8 @@ static int __led_set_brightness_blocking(struct led_classdev *led_cdev, unsigned static void led_timer_function(struct timer_list *t) { - struct led_classdev *led_cdev = from_timer(led_cdev, t, blink_timer); + struct led_classdev *led_cdev = timer_container_of(led_cdev, t, + blink_timer); unsigned long brightness; unsigned long delay; diff --git a/drivers/leds/trigger/ledtrig-activity.c b/drivers/leds/trigger/ledtrig-activity.c index b3ee33aed36e..c973246a57f9 100644 --- a/drivers/leds/trigger/ledtrig-activity.c +++ b/drivers/leds/trigger/ledtrig-activity.c @@ -32,8 +32,8 @@ struct activity_data { static void led_activity_function(struct timer_list *t) { - struct activity_data *activity_data = from_timer(activity_data, t, - timer); + struct activity_data *activity_data = timer_container_of(activity_data, + t, timer); struct led_classdev *led_cdev = activity_data->led_cdev; unsigned int target; unsigned int usage; diff --git a/drivers/leds/trigger/ledtrig-heartbeat.c b/drivers/leds/trigger/ledtrig-heartbeat.c index 393b3ae832f4..40eb61b6d54e 100644 --- a/drivers/leds/trigger/ledtrig-heartbeat.c +++ b/drivers/leds/trigger/ledtrig-heartbeat.c @@ -33,7 +33,7 @@ struct heartbeat_trig_data { static void led_heartbeat_function(struct timer_list *t) { struct heartbeat_trig_data *heartbeat_data = - from_timer(heartbeat_data, t, timer); + timer_container_of(heartbeat_data, t, timer); struct led_classdev *led_cdev; unsigned long brightness = LED_OFF; unsigned long delay = 0; diff --git a/drivers/leds/trigger/ledtrig-pattern.c b/drivers/leds/trigger/ledtrig-pattern.c index 06d052957d37..9af3c18f14f4 100644 --- a/drivers/leds/trigger/ledtrig-pattern.c +++ b/drivers/leds/trigger/ledtrig-pattern.c @@ -150,7 +150,7 @@ static void pattern_trig_timer_common_function(struct pattern_trig_data *data) static void pattern_trig_timer_function(struct timer_list *t) { - struct pattern_trig_data *data = from_timer(data, t, timer); + struct pattern_trig_data *data = timer_container_of(data, t, timer); return pattern_trig_timer_common_function(data); } diff --git a/drivers/leds/trigger/ledtrig-transient.c b/drivers/leds/trigger/ledtrig-transient.c index e103c7ed830b..20f1351464b1 100644 --- a/drivers/leds/trigger/ledtrig-transient.c +++ b/drivers/leds/trigger/ledtrig-transient.c @@ -32,7 +32,7 @@ struct transient_trig_data { static void transient_timer_function(struct timer_list *t) { struct transient_trig_data *transient_data = - from_timer(transient_data, t, timer); + timer_container_of(transient_data, t, timer); struct led_classdev *led_cdev = transient_data->led_cdev; transient_data->activate = 0; diff --git a/drivers/mailbox/mailbox-altera.c b/drivers/mailbox/mailbox-altera.c index 748128661892..17278c2571d3 100644 --- a/drivers/mailbox/mailbox-altera.c +++ b/drivers/mailbox/mailbox-altera.c @@ -130,7 +130,7 @@ static void altera_mbox_rx_data(struct mbox_chan *chan) static void altera_mbox_poll_rx(struct timer_list *t) { - struct altera_mbox *mbox = from_timer(mbox, t, rxpoll_timer); + struct altera_mbox *mbox = timer_container_of(mbox, t, rxpoll_timer); altera_mbox_rx_data(mbox->chan); diff --git a/drivers/md/bcache/stats.c b/drivers/md/bcache/stats.c index d39dec34b7a3..0056106495a7 100644 --- a/drivers/md/bcache/stats.c +++ b/drivers/md/bcache/stats.c @@ -149,7 +149,7 @@ static void scale_stats(struct cache_stats *stats, unsigned long rescale_at) static void scale_accounting(struct timer_list *t) { - struct cache_accounting *acc = from_timer(acc, t, timer); + struct cache_accounting *acc = timer_container_of(acc, t, timer); #define move_stat(name) do { \ unsigned int t = atomic_xchg(&acc->collector.name, 0); \ diff --git a/drivers/md/dm-delay.c b/drivers/md/dm-delay.c index 16d3d454fb0a..4bb6553278c7 100644 --- a/drivers/md/dm-delay.c +++ b/drivers/md/dm-delay.c @@ -56,7 +56,7 @@ struct dm_delay_info { static void handle_delayed_timer(struct timer_list *t) { - struct delay_c *dc = from_timer(dc, t, delay_timer); + struct delay_c *dc = timer_container_of(dc, t, delay_timer); queue_work(dc->kdelayd_wq, &dc->flush_expired_bios); } diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 1f626066e8cc..4395657fa583 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -1540,7 +1540,8 @@ static void sleep_on_endio_wait(struct dm_integrity_c *ic) static void autocommit_fn(struct timer_list *t) { - struct dm_integrity_c *ic = from_timer(ic, t, autocommit_timer); + struct dm_integrity_c *ic = timer_container_of(ic, t, + autocommit_timer); if (likely(!dm_integrity_failed(ic))) queue_work(ic->commit_wq, &ic->commit_work); diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c index 81fec2e1e0ef..aaf4a0a4b0eb 100644 --- a/drivers/md/dm-mpath.c +++ b/drivers/md/dm-mpath.c @@ -790,7 +790,7 @@ static int queue_if_no_path(struct multipath *m, bool f_queue_if_no_path, */ static void queue_if_no_path_timeout_work(struct timer_list *t) { - struct multipath *m = from_timer(m, t, nopath_timer); + struct multipath *m = timer_container_of(m, t, nopath_timer); DMWARN("queue_if_no_path timeout on %s, failing queued IO", dm_table_device_name(m->ti->table)); diff --git a/drivers/md/dm-raid1.c b/drivers/md/dm-raid1.c index 785af4816584..268f734ca9c3 100644 --- a/drivers/md/dm-raid1.c +++ b/drivers/md/dm-raid1.c @@ -103,7 +103,7 @@ static void wakeup_mirrord(void *context) static void delayed_wake_fn(struct timer_list *t) { - struct mirror_set *ms = from_timer(ms, t, timer); + struct mirror_set *ms = timer_container_of(ms, t, timer); clear_bit(0, &ms->timer_pending); wakeup_mirrord(ms); diff --git a/drivers/md/dm-vdo/dedupe.c b/drivers/md/dm-vdo/dedupe.c index 3c58b941e067..4d983092a152 100644 --- a/drivers/md/dm-vdo/dedupe.c +++ b/drivers/md/dm-vdo/dedupe.c @@ -2337,7 +2337,7 @@ static void timeout_index_operations_callback(struct vdo_completion *completion) static void timeout_index_operations(struct timer_list *t) { - struct hash_zone *zone = from_timer(zone, t, timer); + struct hash_zone *zone = timer_container_of(zone, t, timer); if (change_timer_state(zone, DEDUPE_QUERY_TIMER_RUNNING, DEDUPE_QUERY_TIMER_FIRED)) diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index d6a04a57472d..a428e1cacf07 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -706,7 +706,7 @@ static inline void writecache_verify_watermark(struct dm_writecache *wc) static void writecache_max_age_timer(struct timer_list *t) { - struct dm_writecache *wc = from_timer(wc, t, max_age_timer); + struct dm_writecache *wc = timer_container_of(wc, t, max_age_timer); if (!dm_suspended(wc->ti) && !writecache_has_error(wc)) { queue_work(wc->writeback_wq, &wc->writeback_work); @@ -866,7 +866,7 @@ static void writecache_flush_work(struct work_struct *work) static void writecache_autocommit_timer(struct timer_list *t) { - struct dm_writecache *wc = from_timer(wc, t, autocommit_timer); + struct dm_writecache *wc = timer_container_of(wc, t, autocommit_timer); if (!writecache_has_error(wc)) queue_work(wc->writeback_wq, &wc->flush_work); diff --git a/drivers/md/md.c b/drivers/md/md.c index 09042b060086..0f03b21e66e4 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -6048,7 +6048,7 @@ static int add_named_array(const char *val, const struct kernel_param *kp) static void md_safemode_timeout(struct timer_list *t) { - struct mddev *mddev = from_timer(mddev, t, safemode_timer); + struct mddev *mddev = timer_container_of(mddev, t, safemode_timer); mddev->safemode = 1; if (mddev->external) diff --git a/drivers/media/common/saa7146/saa7146_fops.c b/drivers/media/common/saa7146/saa7146_fops.c index 2952678cce45..9d0362a75ecd 100644 --- a/drivers/media/common/saa7146/saa7146_fops.c +++ b/drivers/media/common/saa7146/saa7146_fops.c @@ -153,7 +153,7 @@ void saa7146_buffer_next(struct saa7146_dev *dev, void saa7146_buffer_timeout(struct timer_list *t) { - struct saa7146_dmaqueue *q = from_timer(q, t, timeout); + struct saa7146_dmaqueue *q = timer_container_of(q, t, timeout); struct saa7146_dev *dev = q->dev; unsigned long flags; diff --git a/drivers/media/common/saa7146/saa7146_vbi.c b/drivers/media/common/saa7146/saa7146_vbi.c index 6c324a683be9..1ffcc025d1a4 100644 --- a/drivers/media/common/saa7146/saa7146_vbi.c +++ b/drivers/media/common/saa7146/saa7146_vbi.c @@ -330,7 +330,7 @@ static void vbi_stop(struct saa7146_dev *dev) static void vbi_read_timeout(struct timer_list *t) { - struct saa7146_vv *vv = from_timer(vv, t, vbi_read_timeout); + struct saa7146_vv *vv = timer_container_of(vv, t, vbi_read_timeout); struct saa7146_dev *dev = vv->vbi_dmaq.dev; DEB_VBI("dev:%p\n", dev); diff --git a/drivers/media/dvb-core/dmxdev.c b/drivers/media/dvb-core/dmxdev.c index 1e985f943944..151177e5a06d 100644 --- a/drivers/media/dvb-core/dmxdev.c +++ b/drivers/media/dvb-core/dmxdev.c @@ -352,7 +352,8 @@ static int dvb_dmxdev_set_buffer_size(struct dmxdev_filter *dmxdevfilter, static void dvb_dmxdev_filter_timeout(struct timer_list *t) { - struct dmxdev_filter *dmxdevfilter = from_timer(dmxdevfilter, t, timer); + struct dmxdev_filter *dmxdevfilter = timer_container_of(dmxdevfilter, + t, timer); dmxdevfilter->buffer.error = -ETIMEDOUT; spin_lock_irq(&dmxdevfilter->dev->lock); diff --git a/drivers/media/i2c/tc358743.c b/drivers/media/i2c/tc358743.c index dcef93e1a3bc..3d6703b75bfa 100644 --- a/drivers/media/i2c/tc358743.c +++ b/drivers/media/i2c/tc358743.c @@ -1505,7 +1505,7 @@ static irqreturn_t tc358743_irq_handler(int irq, void *dev_id) static void tc358743_irq_poll_timer(struct timer_list *t) { - struct tc358743_state *state = from_timer(state, t, timer); + struct tc358743_state *state = timer_container_of(state, t, timer); unsigned int msecs; schedule_work(&state->work_i2c_poll); diff --git a/drivers/media/i2c/tvaudio.c b/drivers/media/i2c/tvaudio.c index 42115118a0bd..6267e9ad39c0 100644 --- a/drivers/media/i2c/tvaudio.c +++ b/drivers/media/i2c/tvaudio.c @@ -318,7 +318,7 @@ static int chip_cmd(struct CHIPSTATE *chip, char *name, audiocmd *cmd) static void chip_thread_wake(struct timer_list *t) { - struct CHIPSTATE *chip = from_timer(chip, t, wt); + struct CHIPSTATE *chip = timer_container_of(chip, t, wt); wake_up_process(chip->thread); } diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c index 377a7e7f0499..9ce67f515843 100644 --- a/drivers/media/pci/bt8xx/bttv-driver.c +++ b/drivers/media/pci/bt8xx/bttv-driver.c @@ -2798,7 +2798,7 @@ bttv_irq_wakeup_vbi(struct bttv *btv, struct bttv_buffer *wakeup, static void bttv_irq_timeout(struct timer_list *t) { - struct bttv *btv = from_timer(btv, t, timeout); + struct bttv *btv = timer_container_of(btv, t, timeout); struct bttv_buffer_set old,new; struct bttv_buffer *ovbi; struct bttv_buffer *item; diff --git a/drivers/media/pci/bt8xx/bttv-input.c b/drivers/media/pci/bt8xx/bttv-input.c index 9eb7a5356b4c..84aa269248fd 100644 --- a/drivers/media/pci/bt8xx/bttv-input.c +++ b/drivers/media/pci/bt8xx/bttv-input.c @@ -126,7 +126,7 @@ void bttv_input_irq(struct bttv *btv) static void bttv_input_timer(struct timer_list *t) { - struct bttv_ir *ir = from_timer(ir, t, timer); + struct bttv_ir *ir = timer_container_of(ir, t, timer); struct bttv *btv = ir->btv; if (btv->c.type == BTTV_BOARD_ENLTV_FM_2) @@ -182,7 +182,7 @@ static u32 bttv_rc5_decode(unsigned int code) static void bttv_rc5_timer_end(struct timer_list *t) { - struct bttv_ir *ir = from_timer(ir, t, timer); + struct bttv_ir *ir = timer_container_of(ir, t, timer); ktime_t tv; u32 gap, rc5, scancode; u8 toggle, command, system; diff --git a/drivers/media/pci/cx18/cx18-fileops.c b/drivers/media/pci/cx18/cx18-fileops.c index 7e742733391b..315577d71d95 100644 --- a/drivers/media/pci/cx18/cx18-fileops.c +++ b/drivers/media/pci/cx18/cx18-fileops.c @@ -628,7 +628,7 @@ __poll_t cx18_v4l2_enc_poll(struct file *filp, poll_table *wait) void cx18_vb_timeout(struct timer_list *t) { - struct cx18_stream *s = from_timer(s, t, vb_timeout); + struct cx18_stream *s = timer_container_of(s, t, vb_timeout); /* * Return all of the buffers in error state, so the vbi/vid inode diff --git a/drivers/media/pci/ivtv/ivtv-irq.c b/drivers/media/pci/ivtv/ivtv-irq.c index b3b670b6ef70..748c14e87963 100644 --- a/drivers/media/pci/ivtv/ivtv-irq.c +++ b/drivers/media/pci/ivtv/ivtv-irq.c @@ -1064,7 +1064,7 @@ irqreturn_t ivtv_irq_handler(int irq, void *dev_id) void ivtv_unfinished_dma(struct timer_list *t) { - struct ivtv *itv = from_timer(itv, t, dma_timer); + struct ivtv *itv = timer_container_of(itv, t, dma_timer); if (!test_bit(IVTV_F_I_DMA, &itv->i_flags)) return; diff --git a/drivers/media/pci/netup_unidvb/netup_unidvb_core.c b/drivers/media/pci/netup_unidvb/netup_unidvb_core.c index 16338d13d9c8..9f2ac33cffa7 100644 --- a/drivers/media/pci/netup_unidvb/netup_unidvb_core.c +++ b/drivers/media/pci/netup_unidvb/netup_unidvb_core.c @@ -637,7 +637,7 @@ static void netup_unidvb_queue_cleanup(struct netup_dma *dma) static void netup_unidvb_dma_timeout(struct timer_list *t) { - struct netup_dma *dma = from_timer(dma, t, timeout); + struct netup_dma *dma = timer_container_of(dma, t, timeout); struct netup_unidvb_dev *ndev = dma->ndev; dev_dbg(&ndev->pci_dev->dev, "%s()\n", __func__); diff --git a/drivers/media/pci/saa7134/saa7134-core.c b/drivers/media/pci/saa7134/saa7134-core.c index 84295bdb8ce4..537aa65acdc8 100644 --- a/drivers/media/pci/saa7134/saa7134-core.c +++ b/drivers/media/pci/saa7134/saa7134-core.c @@ -328,7 +328,7 @@ void saa7134_buffer_next(struct saa7134_dev *dev, void saa7134_buffer_timeout(struct timer_list *t) { - struct saa7134_dmaqueue *q = from_timer(q, t, timeout); + struct saa7134_dmaqueue *q = timer_container_of(q, t, timeout); struct saa7134_dev *dev = q->dev; unsigned long flags; diff --git a/drivers/media/pci/saa7134/saa7134-input.c b/drivers/media/pci/saa7134/saa7134-input.c index d7d97c7d4a2b..468dbe8d552f 100644 --- a/drivers/media/pci/saa7134/saa7134-input.c +++ b/drivers/media/pci/saa7134/saa7134-input.c @@ -431,7 +431,7 @@ void saa7134_input_irq(struct saa7134_dev *dev) static void saa7134_input_timer(struct timer_list *t) { - struct saa7134_card_ir *ir = from_timer(ir, t, timer); + struct saa7134_card_ir *ir = timer_container_of(ir, t, timer); struct saa7134_dev *dev = ir->dev->priv; build_key(dev); diff --git a/drivers/media/pci/tw686x/tw686x-core.c b/drivers/media/pci/tw686x/tw686x-core.c index 80bd268926cc..f39e0e34deb6 100644 --- a/drivers/media/pci/tw686x/tw686x-core.c +++ b/drivers/media/pci/tw686x/tw686x-core.c @@ -125,7 +125,7 @@ void tw686x_enable_channel(struct tw686x_dev *dev, unsigned int channel) */ static void tw686x_dma_delay(struct timer_list *t) { - struct tw686x_dev *dev = from_timer(dev, t, dma_delay_timer); + struct tw686x_dev *dev = timer_container_of(dev, t, dma_delay_timer); unsigned long flags; spin_lock_irqsave(&dev->lock, flags); diff --git a/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c b/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c index 9f89bd2620c7..73fdcd362265 100644 --- a/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c +++ b/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c @@ -143,7 +143,7 @@ void s5p_mfc_cleanup_queue(struct list_head *lh, struct vb2_queue *vq) static void s5p_mfc_watchdog(struct timer_list *t) { - struct s5p_mfc_dev *dev = from_timer(dev, t, watchdog_timer); + struct s5p_mfc_dev *dev = timer_container_of(dev, t, watchdog_timer); if (test_bit(0, &dev->hw_lock)) atomic_inc(&dev->watchdog_cnt); diff --git a/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c b/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c index 87a817dda4a9..602c37cbe177 100644 --- a/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c +++ b/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c @@ -62,7 +62,7 @@ static int load_c8sectpfe_fw(struct c8sectpfei *fei); static void c8sectpfe_timer_interrupt(struct timer_list *t) { - struct c8sectpfei *fei = from_timer(fei, t, timer); + struct c8sectpfei *fei = timer_container_of(fei, t, timer); struct channel_info *channel; int chan_num; diff --git a/drivers/media/radio/radio-cadet.c b/drivers/media/radio/radio-cadet.c index 2ddf1dfa0522..5110754e1a31 100644 --- a/drivers/media/radio/radio-cadet.c +++ b/drivers/media/radio/radio-cadet.c @@ -284,7 +284,7 @@ static bool cadet_has_rds_data(struct cadet *dev) static void cadet_handler(struct timer_list *t) { - struct cadet *dev = from_timer(dev, t, readtimer); + struct cadet *dev = timer_container_of(dev, t, readtimer); /* Service the RDS fifo */ if (mutex_trylock(&dev->lock)) { diff --git a/drivers/media/rc/ene_ir.c b/drivers/media/rc/ene_ir.c index 9435cba3f4d9..d6c54a3bccc2 100644 --- a/drivers/media/rc/ene_ir.c +++ b/drivers/media/rc/ene_ir.c @@ -659,7 +659,7 @@ exit: /* timer to simulate tx done interrupt */ static void ene_tx_irqsim(struct timer_list *t) { - struct ene_device *dev = from_timer(dev, t, tx_sim_timer); + struct ene_device *dev = timer_container_of(dev, t, tx_sim_timer); unsigned long flags; spin_lock_irqsave(&dev->hw_lock, flags); diff --git a/drivers/media/rc/igorplugusb.c b/drivers/media/rc/igorplugusb.c index bfe86588c69b..e034c93d57cf 100644 --- a/drivers/media/rc/igorplugusb.c +++ b/drivers/media/rc/igorplugusb.c @@ -131,7 +131,7 @@ static void igorplugusb_cmd(struct igorplugusb *ir, int cmd) static void igorplugusb_timer(struct timer_list *t) { - struct igorplugusb *ir = from_timer(ir, t, timer); + struct igorplugusb *ir = timer_container_of(ir, t, timer); igorplugusb_cmd(ir, GET_INFRACODE); } diff --git a/drivers/media/rc/img-ir/img-ir-hw.c b/drivers/media/rc/img-ir/img-ir-hw.c index da89ddf771c3..63f6f5b36838 100644 --- a/drivers/media/rc/img-ir/img-ir-hw.c +++ b/drivers/media/rc/img-ir/img-ir-hw.c @@ -865,7 +865,7 @@ static void img_ir_handle_data(struct img_ir_priv *priv, u32 len, u64 raw) /* timer function to end waiting for repeat. */ static void img_ir_end_timer(struct timer_list *t) { - struct img_ir_priv *priv = from_timer(priv, t, hw.end_timer); + struct img_ir_priv *priv = timer_container_of(priv, t, hw.end_timer); spin_lock_irq(&priv->lock); img_ir_end_repeat(priv); @@ -879,7 +879,8 @@ static void img_ir_end_timer(struct timer_list *t) */ static void img_ir_suspend_timer(struct timer_list *t) { - struct img_ir_priv *priv = from_timer(priv, t, hw.suspend_timer); + struct img_ir_priv *priv = timer_container_of(priv, t, + hw.suspend_timer); spin_lock_irq(&priv->lock); /* diff --git a/drivers/media/rc/img-ir/img-ir-raw.c b/drivers/media/rc/img-ir/img-ir-raw.c index 669f3309e237..92fb7b555a0f 100644 --- a/drivers/media/rc/img-ir/img-ir-raw.c +++ b/drivers/media/rc/img-ir/img-ir-raw.c @@ -65,7 +65,7 @@ void img_ir_isr_raw(struct img_ir_priv *priv, u32 irq_status) */ static void img_ir_echo_timer(struct timer_list *t) { - struct img_ir_priv *priv = from_timer(priv, t, raw.timer); + struct img_ir_priv *priv = timer_container_of(priv, t, raw.timer); spin_lock_irq(&priv->lock); diff --git a/drivers/media/rc/imon.c b/drivers/media/rc/imon.c index cb6f36ebe5c8..f5221b018808 100644 --- a/drivers/media/rc/imon.c +++ b/drivers/media/rc/imon.c @@ -1091,7 +1091,7 @@ static void usb_tx_callback(struct urb *urb) */ static void imon_touch_display_timeout(struct timer_list *t) { - struct imon_context *ictx = from_timer(ictx, t, ttimer); + struct imon_context *ictx = timer_container_of(ictx, t, ttimer); if (ictx->display_type != IMON_DISPLAY_TYPE_VGA) return; diff --git a/drivers/media/rc/ir-mce_kbd-decoder.c b/drivers/media/rc/ir-mce_kbd-decoder.c index 817030fb50c9..bb2d7c37c263 100644 --- a/drivers/media/rc/ir-mce_kbd-decoder.c +++ b/drivers/media/rc/ir-mce_kbd-decoder.c @@ -109,7 +109,8 @@ static unsigned char kbd_keycodes[256] = { static void mce_kbd_rx_timeout(struct timer_list *t) { - struct ir_raw_event_ctrl *raw = from_timer(raw, t, mce_kbd.rx_timeout); + struct ir_raw_event_ctrl *raw = timer_container_of(raw, t, + mce_kbd.rx_timeout); unsigned char maskcode; unsigned long flags; int i; diff --git a/drivers/media/rc/rc-ir-raw.c b/drivers/media/rc/rc-ir-raw.c index aa4ac43c66fa..5dafe11f61c6 100644 --- a/drivers/media/rc/rc-ir-raw.c +++ b/drivers/media/rc/rc-ir-raw.c @@ -552,7 +552,8 @@ EXPORT_SYMBOL(ir_raw_encode_scancode); */ static void ir_raw_edge_handle(struct timer_list *t) { - struct ir_raw_event_ctrl *raw = from_timer(raw, t, edge_handle); + struct ir_raw_event_ctrl *raw = timer_container_of(raw, t, + edge_handle); struct rc_dev *dev = raw->dev; unsigned long flags; ktime_t interval; diff --git a/drivers/media/rc/rc-main.c b/drivers/media/rc/rc-main.c index e46358fb8ac0..b9bf5cdcde4a 100644 --- a/drivers/media/rc/rc-main.c +++ b/drivers/media/rc/rc-main.c @@ -674,7 +674,7 @@ EXPORT_SYMBOL_GPL(rc_keyup); */ static void ir_timer_keyup(struct timer_list *t) { - struct rc_dev *dev = from_timer(dev, t, timer_keyup); + struct rc_dev *dev = timer_container_of(dev, t, timer_keyup); unsigned long flags; /* @@ -703,7 +703,7 @@ static void ir_timer_keyup(struct timer_list *t) */ static void ir_timer_repeat(struct timer_list *t) { - struct rc_dev *dev = from_timer(dev, t, timer_repeat); + struct rc_dev *dev = timer_container_of(dev, t, timer_repeat); struct input_dev *input = dev->input_dev; unsigned long flags; diff --git a/drivers/media/usb/au0828/au0828-dvb.c b/drivers/media/usb/au0828/au0828-dvb.c index 3666f4452d11..5d0447ff7d06 100644 --- a/drivers/media/usb/au0828/au0828-dvb.c +++ b/drivers/media/usb/au0828/au0828-dvb.c @@ -97,7 +97,7 @@ static void au0828_restart_dvb_streaming(struct work_struct *work); static void au0828_bulk_timeout(struct timer_list *t) { - struct au0828_dev *dev = from_timer(dev, t, bulk_timeout); + struct au0828_dev *dev = timer_container_of(dev, t, bulk_timeout); dprintk(1, "%s called\n", __func__); dev->bulk_timeout_running = 0; diff --git a/drivers/media/usb/au0828/au0828-video.c b/drivers/media/usb/au0828/au0828-video.c index 33d1fad0f7b8..e5dff969ed57 100644 --- a/drivers/media/usb/au0828/au0828-video.c +++ b/drivers/media/usb/au0828/au0828-video.c @@ -948,7 +948,7 @@ int au0828_analog_unregister(struct au0828_dev *dev) such as tvtime from hanging) */ static void au0828_vid_buffer_timeout(struct timer_list *t) { - struct au0828_dev *dev = from_timer(dev, t, vid_timeout); + struct au0828_dev *dev = timer_container_of(dev, t, vid_timeout); struct au0828_dmaqueue *dma_q = &dev->vidq; struct au0828_buffer *buf; unsigned char *vid_data; @@ -972,7 +972,7 @@ static void au0828_vid_buffer_timeout(struct timer_list *t) static void au0828_vbi_buffer_timeout(struct timer_list *t) { - struct au0828_dev *dev = from_timer(dev, t, vbi_timeout); + struct au0828_dev *dev = timer_container_of(dev, t, vbi_timeout); struct au0828_dmaqueue *dma_q = &dev->vbiq; struct au0828_buffer *buf; unsigned char *vbi_data; diff --git a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c index e23b0de1e0aa..f21c2806eb9f 100644 --- a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c +++ b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c @@ -3562,7 +3562,7 @@ struct hdw_timer { static void pvr2_ctl_timeout(struct timer_list *t) { - struct hdw_timer *timer = from_timer(timer, t, timer); + struct hdw_timer *timer = timer_container_of(timer, t, timer); struct pvr2_hdw *hdw = timer->hdw; if (hdw->ctl_write_pend_flag || hdw->ctl_read_pend_flag) { @@ -4421,7 +4421,7 @@ static int state_eval_encoder_run(struct pvr2_hdw *hdw) /* Timeout function for quiescent timer. */ static void pvr2_hdw_quiescent_timeout(struct timer_list *t) { - struct pvr2_hdw *hdw = from_timer(hdw, t, quiescent_timer); + struct pvr2_hdw *hdw = timer_container_of(hdw, t, quiescent_timer); hdw->state_decoder_quiescent = !0; trace_stbit("state_decoder_quiescent",hdw->state_decoder_quiescent); hdw->state_stale = !0; @@ -4432,7 +4432,8 @@ static void pvr2_hdw_quiescent_timeout(struct timer_list *t) /* Timeout function for decoder stabilization timer. */ static void pvr2_hdw_decoder_stabilization_timeout(struct timer_list *t) { - struct pvr2_hdw *hdw = from_timer(hdw, t, decoder_stabilization_timer); + struct pvr2_hdw *hdw = timer_container_of(hdw, t, + decoder_stabilization_timer); hdw->state_decoder_ready = !0; trace_stbit("state_decoder_ready", hdw->state_decoder_ready); hdw->state_stale = !0; @@ -4443,7 +4444,7 @@ static void pvr2_hdw_decoder_stabilization_timeout(struct timer_list *t) /* Timeout function for encoder wait timer. */ static void pvr2_hdw_encoder_wait_timeout(struct timer_list *t) { - struct pvr2_hdw *hdw = from_timer(hdw, t, encoder_wait_timer); + struct pvr2_hdw *hdw = timer_container_of(hdw, t, encoder_wait_timer); hdw->state_encoder_waitok = !0; trace_stbit("state_encoder_waitok",hdw->state_encoder_waitok); hdw->state_stale = !0; @@ -4454,7 +4455,7 @@ static void pvr2_hdw_encoder_wait_timeout(struct timer_list *t) /* Timeout function for encoder run timer. */ static void pvr2_hdw_encoder_run_timeout(struct timer_list *t) { - struct pvr2_hdw *hdw = from_timer(hdw, t, encoder_run_timer); + struct pvr2_hdw *hdw = timer_container_of(hdw, t, encoder_run_timer); if (!hdw->state_encoder_runok) { hdw->state_encoder_runok = !0; trace_stbit("state_encoder_runok",hdw->state_encoder_runok); diff --git a/drivers/media/usb/s2255/s2255drv.c b/drivers/media/usb/s2255/s2255drv.c index 899a7a67e2ba..8332f2c5aed7 100644 --- a/drivers/media/usb/s2255/s2255drv.c +++ b/drivers/media/usb/s2255/s2255drv.c @@ -471,7 +471,7 @@ static void s2255_reset_dsppower(struct s2255_dev *dev) */ static void s2255_timer(struct timer_list *t) { - struct s2255_dev *dev = from_timer(dev, t, timer); + struct s2255_dev *dev = timer_container_of(dev, t, timer); struct s2255_fw *data = dev->fw_data; if (usb_submit_urb(data->fw_urb, GFP_ATOMIC) < 0) { pr_err("s2255: can't submit urb\n"); diff --git a/drivers/memory/tegra/tegra210-emc-core.c b/drivers/memory/tegra/tegra210-emc-core.c index e63f62690571..e96ca4157d48 100644 --- a/drivers/memory/tegra/tegra210-emc-core.c +++ b/drivers/memory/tegra/tegra210-emc-core.c @@ -558,7 +558,7 @@ tegra210_emc_table_register_offsets = { static void tegra210_emc_train(struct timer_list *timer) { - struct tegra210_emc *emc = from_timer(emc, timer, training); + struct tegra210_emc *emc = timer_container_of(emc, timer, training); unsigned long flags; if (!emc->last) @@ -614,7 +614,8 @@ static unsigned int tegra210_emc_get_temperature(struct tegra210_emc *emc) static void tegra210_emc_poll_refresh(struct timer_list *timer) { - struct tegra210_emc *emc = from_timer(emc, timer, refresh_timer); + struct tegra210_emc *emc = timer_container_of(emc, timer, + refresh_timer); unsigned int temperature; if (!emc->debugfs.temperature) diff --git a/drivers/memstick/core/ms_block.c b/drivers/memstick/core/ms_block.c index 7dc2c9987982..d34892782f6e 100644 --- a/drivers/memstick/core/ms_block.c +++ b/drivers/memstick/core/ms_block.c @@ -1498,7 +1498,7 @@ static int msb_ftl_scan(struct msb_data *msb) static void msb_cache_flush_timer(struct timer_list *t) { - struct msb_data *msb = from_timer(msb, t, cache_flush_timer); + struct msb_data *msb = timer_container_of(msb, t, cache_flush_timer); msb->need_flush_cache = true; queue_work(msb->io_queue, &msb->io_work); diff --git a/drivers/memstick/host/jmb38x_ms.c b/drivers/memstick/host/jmb38x_ms.c index a5a9bb3f16be..cddddb3a5a27 100644 --- a/drivers/memstick/host/jmb38x_ms.c +++ b/drivers/memstick/host/jmb38x_ms.c @@ -590,7 +590,7 @@ static irqreturn_t jmb38x_ms_isr(int irq, void *dev_id) static void jmb38x_ms_abort(struct timer_list *t) { - struct jmb38x_ms_host *host = from_timer(host, t, timer); + struct jmb38x_ms_host *host = timer_container_of(host, t, timer); struct memstick_host *msh = host->msh; unsigned long flags; diff --git a/drivers/memstick/host/r592.c b/drivers/memstick/host/r592.c index 488ef8eecb26..605b2265536f 100644 --- a/drivers/memstick/host/r592.c +++ b/drivers/memstick/host/r592.c @@ -614,7 +614,7 @@ static void r592_update_card_detect(struct r592_device *dev) /* Timer routine that fires 1 second after last card detection event, */ static void r592_detect_timer(struct timer_list *t) { - struct r592_device *dev = from_timer(dev, t, detect_timer); + struct r592_device *dev = timer_container_of(dev, t, detect_timer); r592_update_card_detect(dev); memstick_detect_change(dev->host); } diff --git a/drivers/memstick/host/tifm_ms.c b/drivers/memstick/host/tifm_ms.c index 676348eee015..db7f3a088fb0 100644 --- a/drivers/memstick/host/tifm_ms.c +++ b/drivers/memstick/host/tifm_ms.c @@ -535,7 +535,7 @@ static int tifm_ms_set_param(struct memstick_host *msh, static void tifm_ms_abort(struct timer_list *t) { - struct tifm_ms *host = from_timer(host, t, timer); + struct tifm_ms *host = timer_container_of(host, t, timer); dev_dbg(&host->dev->dev, "status %x\n", readl(host->dev->addr + SOCK_MS_STATUS)); diff --git a/drivers/misc/bcm-vk/bcm_vk_tty.c b/drivers/misc/bcm-vk/bcm_vk_tty.c index 44a2dd80054d..e6c42b772e96 100644 --- a/drivers/misc/bcm-vk/bcm_vk_tty.c +++ b/drivers/misc/bcm-vk/bcm_vk_tty.c @@ -43,7 +43,7 @@ struct bcm_vk_tty_chan { static void bcm_vk_tty_poll(struct timer_list *t) { - struct bcm_vk *vk = from_timer(vk, t, serial_timer); + struct bcm_vk *vk = timer_container_of(vk, t, serial_timer); queue_work(vk->tty_wq_thread, &vk->tty_wq_work); mod_timer(&vk->serial_timer, jiffies + SERIAL_TIMER_VALUE); diff --git a/drivers/misc/cardreader/rtsx_usb.c b/drivers/misc/cardreader/rtsx_usb.c index 7314c8d9ae75..148107a4547c 100644 --- a/drivers/misc/cardreader/rtsx_usb.c +++ b/drivers/misc/cardreader/rtsx_usb.c @@ -31,7 +31,7 @@ static const struct mfd_cell rtsx_usb_cells[] = { static void rtsx_usb_sg_timed_out(struct timer_list *t) { - struct rtsx_ucr *ucr = from_timer(ucr, t, sg_timer); + struct rtsx_ucr *ucr = timer_container_of(ucr, t, sg_timer); dev_dbg(&ucr->pusb_intf->dev, "%s: sg transfer timed out", __func__); usb_sg_cancel(&ucr->current_sg); diff --git a/drivers/misc/sgi-xp/xpc_main.c b/drivers/misc/sgi-xp/xpc_main.c index 697a008c14d3..9fe816bf3957 100644 --- a/drivers/misc/sgi-xp/xpc_main.c +++ b/drivers/misc/sgi-xp/xpc_main.c @@ -164,7 +164,8 @@ struct xpc_arch_operations xpc_arch_ops; static void xpc_timeout_partition_disengage(struct timer_list *t) { - struct xpc_partition *part = from_timer(part, t, disengage_timer); + struct xpc_partition *part = timer_container_of(part, t, + disengage_timer); DBUG_ON(time_is_after_jiffies(part->disengage_timeout)); diff --git a/drivers/mmc/core/host.c b/drivers/mmc/core/host.c index dacb5bd9bb71..f14671ea5716 100644 --- a/drivers/mmc/core/host.c +++ b/drivers/mmc/core/host.c @@ -212,7 +212,7 @@ out: static void mmc_retune_timer(struct timer_list *t) { - struct mmc_host *host = from_timer(host, t, retune_timer); + struct mmc_host *host = timer_container_of(host, t, retune_timer); mmc_retune_needed(host); } diff --git a/drivers/mmc/host/atmel-mci.c b/drivers/mmc/host/atmel-mci.c index 14e981b834b6..0e0666c0bb6e 100644 --- a/drivers/mmc/host/atmel-mci.c +++ b/drivers/mmc/host/atmel-mci.c @@ -714,7 +714,7 @@ static inline unsigned int atmci_convert_chksize(struct atmel_mci *host, static void atmci_timeout_timer(struct timer_list *t) { - struct atmel_mci *host = from_timer(host, t, timer); + struct atmel_mci *host = timer_container_of(host, t, timer); struct device *dev = host->dev; dev_dbg(dev, "software timeout\n"); @@ -1652,7 +1652,8 @@ static void atmci_command_complete(struct atmel_mci *host, static void atmci_detect_change(struct timer_list *t) { - struct atmel_mci_slot *slot = from_timer(slot, t, detect_timer); + struct atmel_mci_slot *slot = timer_container_of(slot, t, + detect_timer); bool present; bool present_old; diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c index 2bfcc47dcf3e..988492237707 100644 --- a/drivers/mmc/host/dw_mmc.c +++ b/drivers/mmc/host/dw_mmc.c @@ -3179,7 +3179,7 @@ no_dma: static void dw_mci_cmd11_timer(struct timer_list *t) { - struct dw_mci *host = from_timer(host, t, cmd11_timer); + struct dw_mci *host = timer_container_of(host, t, cmd11_timer); if (host->state != STATE_SENDING_CMD11) { dev_warn(host->dev, "Unexpected CMD11 timeout\n"); @@ -3193,7 +3193,7 @@ static void dw_mci_cmd11_timer(struct timer_list *t) static void dw_mci_cto_timer(struct timer_list *t) { - struct dw_mci *host = from_timer(host, t, cto_timer); + struct dw_mci *host = timer_container_of(host, t, cto_timer); unsigned long irqflags; u32 pending; @@ -3248,7 +3248,7 @@ exit: static void dw_mci_dto_timer(struct timer_list *t) { - struct dw_mci *host = from_timer(host, t, dto_timer); + struct dw_mci *host = timer_container_of(host, t, dto_timer); unsigned long irqflags; u32 pending; diff --git a/drivers/mmc/host/jz4740_mmc.c b/drivers/mmc/host/jz4740_mmc.c index bd1662e275d4..0fbbf57db52e 100644 --- a/drivers/mmc/host/jz4740_mmc.c +++ b/drivers/mmc/host/jz4740_mmc.c @@ -641,7 +641,8 @@ poll_timeout: static void jz4740_mmc_timeout(struct timer_list *t) { - struct jz4740_mmc_host *host = from_timer(host, t, timeout_timer); + struct jz4740_mmc_host *host = timer_container_of(host, t, + timeout_timer); if (!test_and_clear_bit(0, &host->waiting)) return; diff --git a/drivers/mmc/host/meson-mx-sdio.c b/drivers/mmc/host/meson-mx-sdio.c index e0ae5a0c9670..8a49c32fd3f9 100644 --- a/drivers/mmc/host/meson-mx-sdio.c +++ b/drivers/mmc/host/meson-mx-sdio.c @@ -467,7 +467,8 @@ static irqreturn_t meson_mx_mmc_irq_thread(int irq, void *irq_data) static void meson_mx_mmc_timeout(struct timer_list *t) { - struct meson_mx_mmc_host *host = from_timer(host, t, cmd_timeout); + struct meson_mx_mmc_host *host = timer_container_of(host, t, + cmd_timeout); unsigned long irqflags; u32 irqc; diff --git a/drivers/mmc/host/mvsdio.c b/drivers/mmc/host/mvsdio.c index 912ffacbad88..101f36de7b63 100644 --- a/drivers/mmc/host/mvsdio.c +++ b/drivers/mmc/host/mvsdio.c @@ -509,7 +509,7 @@ static irqreturn_t mvsd_irq(int irq, void *dev) static void mvsd_timeout_timer(struct timer_list *t) { - struct mvsd_host *host = from_timer(host, t, timer); + struct mvsd_host *host = timer_container_of(host, t, timer); void __iomem *iobase = host->base; struct mmc_request *mrq; unsigned long flags; diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c index 95d8d40a06a8..e588e24256cc 100644 --- a/drivers/mmc/host/mxcmmc.c +++ b/drivers/mmc/host/mxcmmc.c @@ -955,7 +955,7 @@ static bool filter(struct dma_chan *chan, void *param) static void mxcmci_watchdog(struct timer_list *t) { - struct mxcmci_host *host = from_timer(host, t, watchdog); + struct mxcmci_host *host = timer_container_of(host, t, watchdog); struct mmc_request *req = host->req; unsigned int stat = mxcmci_readl(host, MMC_REG_STATUS); diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c index c50617d03709..c2be0f04439d 100644 --- a/drivers/mmc/host/omap.c +++ b/drivers/mmc/host/omap.c @@ -639,7 +639,8 @@ static void mmc_omap_abort_command(struct work_struct *work) static void mmc_omap_cmd_timer(struct timer_list *t) { - struct mmc_omap_host *host = from_timer(host, t, cmd_abort_timer); + struct mmc_omap_host *host = timer_container_of(host, t, + cmd_abort_timer); unsigned long flags; spin_lock_irqsave(&host->slot_lock, flags); @@ -655,7 +656,7 @@ mmc_omap_cmd_timer(struct timer_list *t) static void mmc_omap_clk_timer(struct timer_list *t) { - struct mmc_omap_host *host = from_timer(host, t, clk_timer); + struct mmc_omap_host *host = timer_container_of(host, t, clk_timer); mmc_omap_fclk_enable(host, 0); } @@ -879,7 +880,7 @@ void omap_mmc_notify_cover_event(struct device *dev, int num, int is_closed) static void mmc_omap_cover_timer(struct timer_list *t) { - struct mmc_omap_slot *slot = from_timer(slot, t, cover_timer); + struct mmc_omap_slot *slot = timer_container_of(slot, t, cover_timer); queue_work(system_bh_wq, &slot->cover_bh_work); } diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c index 32fa0b2bb912..f008167d1863 100644 --- a/drivers/mmc/host/sdhci.c +++ b/drivers/mmc/host/sdhci.c @@ -3245,7 +3245,7 @@ static void sdhci_timeout_timer(struct timer_list *t) struct sdhci_host *host; unsigned long flags; - host = from_timer(host, t, timer); + host = timer_container_of(host, t, timer); spin_lock_irqsave(&host->lock, flags); @@ -3267,7 +3267,7 @@ static void sdhci_timeout_data_timer(struct timer_list *t) struct sdhci_host *host; unsigned long flags; - host = from_timer(host, t, data_timer); + host = timer_container_of(host, t, data_timer); spin_lock_irqsave(&host->lock, flags); diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c index 713223f2d377..5e5ec92f80e6 100644 --- a/drivers/mmc/host/tifm_sd.c +++ b/drivers/mmc/host/tifm_sd.c @@ -777,7 +777,7 @@ static void tifm_sd_end_cmd(struct work_struct *t) static void tifm_sd_abort(struct timer_list *t) { - struct tifm_sd *host = from_timer(host, t, timer); + struct tifm_sd *host = timer_container_of(host, t, timer); pr_err("%s : card failed to respond for a long period of time " "(%x, %x)\n", diff --git a/drivers/mmc/host/via-sdmmc.c b/drivers/mmc/host/via-sdmmc.c index 909d80a02824..9903966c2f54 100644 --- a/drivers/mmc/host/via-sdmmc.c +++ b/drivers/mmc/host/via-sdmmc.c @@ -937,7 +937,7 @@ static void via_sdc_timeout(struct timer_list *t) struct via_crdr_mmc_host *sdhost; unsigned long flags; - sdhost = from_timer(sdhost, t, timer); + sdhost = timer_container_of(sdhost, t, timer); spin_lock_irqsave(&sdhost->lock, flags); diff --git a/drivers/mmc/host/vub300.c b/drivers/mmc/host/vub300.c index dd71e5b8e1a5..f498fe11ecdf 100644 --- a/drivers/mmc/host/vub300.c +++ b/drivers/mmc/host/vub300.c @@ -740,8 +740,8 @@ static void vub300_deadwork_thread(struct work_struct *work) static void vub300_inactivity_timer_expired(struct timer_list *t) { /* softirq */ - struct vub300_mmc_host *vub300 = from_timer(vub300, t, - inactivity_timer); + struct vub300_mmc_host *vub300 = timer_container_of(vub300, t, + inactivity_timer); if (!vub300->interface) { kref_put(&vub300->kref, vub300_delete); } else if (vub300->cmd) { @@ -1180,8 +1180,8 @@ static void send_command(struct vub300_mmc_host *vub300) */ static void vub300_sg_timed_out(struct timer_list *t) { - struct vub300_mmc_host *vub300 = from_timer(vub300, t, - sg_transfer_timer); + struct vub300_mmc_host *vub300 = timer_container_of(vub300, t, + sg_transfer_timer); vub300->usb_timed_out = 1; usb_sg_cancel(&vub300->sg_request); usb_unlink_urb(vub300->command_out_urb); diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c index d5974b355a5a..2ae787d966de 100644 --- a/drivers/mmc/host/wbsd.c +++ b/drivers/mmc/host/wbsd.c @@ -947,7 +947,7 @@ static const struct mmc_host_ops wbsd_ops = { static void wbsd_reset_ignore(struct timer_list *t) { - struct wbsd_host *host = from_timer(host, t, ignore_timer); + struct wbsd_host *host = timer_container_of(host, t, ignore_timer); BUG_ON(host == NULL); diff --git a/drivers/most/most_usb.c b/drivers/most/most_usb.c index 2199ba821922..cf5be9c449a5 100644 --- a/drivers/most/most_usb.c +++ b/drivers/most/most_usb.c @@ -667,7 +667,7 @@ static void hdm_request_netinfo(struct most_interface *iface, int channel, */ static void link_stat_timer_handler(struct timer_list *t) { - struct most_dev *mdev = from_timer(mdev, t, link_stat_timer); + struct most_dev *mdev = timer_container_of(mdev, t, link_stat_timer); schedule_work(&mdev->poll_work_obj); mdev->link_stat_timer.expires = jiffies + (2 * HZ); diff --git a/drivers/mtd/sm_ftl.c b/drivers/mtd/sm_ftl.c index d28d4f1790f5..abc7b186353f 100644 --- a/drivers/mtd/sm_ftl.c +++ b/drivers/mtd/sm_ftl.c @@ -993,7 +993,7 @@ restart: /* flush timer, runs a second after last write */ static void sm_cache_flush_timer(struct timer_list *t) { - struct sm_ftl *ftl = from_timer(ftl, t, timer); + struct sm_ftl *ftl = timer_container_of(ftl, t, timer); queue_work(cache_flush_workqueue, &ftl->flush_work); } diff --git a/drivers/net/arcnet/arcnet.c b/drivers/net/arcnet/arcnet.c index 602e6e1adf00..882972604c82 100644 --- a/drivers/net/arcnet/arcnet.c +++ b/drivers/net/arcnet/arcnet.c @@ -382,7 +382,7 @@ static void arcdev_setup(struct net_device *dev) static void arcnet_timer(struct timer_list *t) { - struct arcnet_local *lp = from_timer(lp, t, timer); + struct arcnet_local *lp = timer_container_of(lp, t, timer); struct net_device *dev = lp->dev; spin_lock_irq(&lp->lock); diff --git a/drivers/net/can/grcan.c b/drivers/net/can/grcan.c index adf3970f070f..c5784d9779ef 100644 --- a/drivers/net/can/grcan.c +++ b/drivers/net/can/grcan.c @@ -806,7 +806,7 @@ static irqreturn_t grcan_interrupt(int irq, void *dev_id) */ static void grcan_running_reset(struct timer_list *t) { - struct grcan_priv *priv = from_timer(priv, t, rr_timer); + struct grcan_priv *priv = timer_container_of(priv, t, rr_timer); struct net_device *dev = priv->dev; struct grcan_registers __iomem *regs = priv->regs; unsigned long flags; @@ -897,7 +897,7 @@ static inline void grcan_reset_timer(struct timer_list *timer, __u32 bitrate) /* Disable channels and schedule a running reset */ static void grcan_initiate_running_reset(struct timer_list *t) { - struct grcan_priv *priv = from_timer(priv, t, hang_timer); + struct grcan_priv *priv = timer_container_of(priv, t, hang_timer); struct net_device *dev = priv->dev; struct grcan_registers __iomem *regs = priv->regs; unsigned long flags; diff --git a/drivers/net/can/kvaser_pciefd.c b/drivers/net/can/kvaser_pciefd.c index 52301511ed1b..09510663988c 100644 --- a/drivers/net/can/kvaser_pciefd.c +++ b/drivers/net/can/kvaser_pciefd.c @@ -937,7 +937,8 @@ static int kvaser_pciefd_get_berr_counter(const struct net_device *ndev, static void kvaser_pciefd_bec_poll_timer(struct timer_list *data) { - struct kvaser_pciefd_can *can = from_timer(can, data, bec_poll_timer); + struct kvaser_pciefd_can *can = timer_container_of(can, data, + bec_poll_timer); kvaser_pciefd_enable_err_gen(can); kvaser_pciefd_request_status(can); diff --git a/drivers/net/can/sja1000/peak_pcmcia.c b/drivers/net/can/sja1000/peak_pcmcia.c index 6c7b1c58f85f..ce18e9e56059 100644 --- a/drivers/net/can/sja1000/peak_pcmcia.c +++ b/drivers/net/can/sja1000/peak_pcmcia.c @@ -374,7 +374,7 @@ static inline void pcan_set_can_power(struct pcan_pccard *card, int onoff) */ static void pcan_led_timer(struct timer_list *t) { - struct pcan_pccard *card = from_timer(card, t, led_timer); + struct pcan_pccard *card = timer_container_of(card, t, led_timer); struct net_device *netdev; int i, up_count = 0; u8 ccr; diff --git a/drivers/net/can/usb/peak_usb/pcan_usb.c b/drivers/net/can/usb/peak_usb/pcan_usb.c index c75df1755b3b..6b293a9056c2 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb.c @@ -319,7 +319,7 @@ static int pcan_usb_write_mode(struct peak_usb_device *dev, u8 onoff) */ static void pcan_usb_restart(struct timer_list *t) { - struct pcan_usb *pdev = from_timer(pdev, t, restart_timer); + struct pcan_usb *pdev = timer_container_of(pdev, t, restart_timer); struct peak_usb_device *dev = &pdev->dev; /* notify candev and netdev */ diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c index 862bdccb7439..dc2f4adac9bc 100644 --- a/drivers/net/dsa/b53/b53_common.c +++ b/drivers/net/dsa/b53/b53_common.c @@ -2034,9 +2034,6 @@ int b53_br_join(struct dsa_switch *ds, int port, struct dsa_bridge bridge, b53_get_vlan_entry(dev, pvid, vl); vl->members &= ~BIT(port); - if (vl->members == BIT(cpu_port)) - vl->members &= ~BIT(cpu_port); - vl->untag = vl->members; b53_set_vlan_entry(dev, pvid, vl); } @@ -2115,8 +2112,7 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct dsa_bridge bridge) } b53_get_vlan_entry(dev, pvid, vl); - vl->members |= BIT(port) | BIT(cpu_port); - vl->untag |= BIT(port) | BIT(cpu_port); + vl->members |= BIT(port); b53_set_vlan_entry(dev, pvid, vl); } } diff --git a/drivers/net/dsa/mv88e6xxx/phy.c b/drivers/net/dsa/mv88e6xxx/phy.c index b6d249eb64e7..4e7827ee684a 100644 --- a/drivers/net/dsa/mv88e6xxx/phy.c +++ b/drivers/net/dsa/mv88e6xxx/phy.c @@ -182,7 +182,7 @@ static void mv88e6xxx_phy_ppu_reenable_work(struct work_struct *ugly) static void mv88e6xxx_phy_ppu_reenable_timer(struct timer_list *t) { - struct mv88e6xxx_chip *chip = from_timer(chip, t, ppu_timer); + struct mv88e6xxx_chip *chip = timer_container_of(chip, t, ppu_timer); schedule_work(&chip->ppu_work); } diff --git a/drivers/net/eql.c b/drivers/net/eql.c index 5889759b8d83..9ba10efd3794 100644 --- a/drivers/net/eql.c +++ b/drivers/net/eql.c @@ -143,7 +143,7 @@ static void eql_kill_one_slave(slave_queue_t *queue, slave_t *slave); static void eql_timer(struct timer_list *t) { - equalizer_t *eql = from_timer(eql, t, timer); + equalizer_t *eql = timer_container_of(eql, t, timer); struct list_head *this, *tmp, *head; spin_lock(&eql->queue.lock); diff --git a/drivers/net/ethernet/3com/3c515.c b/drivers/net/ethernet/3com/3c515.c index 8ba2ed87fe7c..ecdea58e6a21 100644 --- a/drivers/net/ethernet/3com/3c515.c +++ b/drivers/net/ethernet/3com/3c515.c @@ -859,7 +859,7 @@ static int corkscrew_open(struct net_device *dev) static void corkscrew_timer(struct timer_list *t) { #ifdef AUTOMEDIA - struct corkscrew_private *vp = from_timer(vp, t, timer); + struct corkscrew_private *vp = timer_container_of(vp, t, timer); struct net_device *dev = vp->our_dev; int ioaddr = dev->base_addr; unsigned long flags; diff --git a/drivers/net/ethernet/3com/3c574_cs.c b/drivers/net/ethernet/3com/3c574_cs.c index b295d528a237..1f2070497a75 100644 --- a/drivers/net/ethernet/3com/3c574_cs.c +++ b/drivers/net/ethernet/3com/3c574_cs.c @@ -858,7 +858,7 @@ static irqreturn_t el3_interrupt(int irq, void *dev_id) */ static void media_check(struct timer_list *t) { - struct el3_private *lp = from_timer(lp, t, media); + struct el3_private *lp = timer_container_of(lp, t, media); struct net_device *dev = lp->p_dev->priv; unsigned int ioaddr = dev->base_addr; unsigned long flags; diff --git a/drivers/net/ethernet/3com/3c589_cs.c b/drivers/net/ethernet/3com/3c589_cs.c index ff331a3bde73..ea49be43b8c3 100644 --- a/drivers/net/ethernet/3com/3c589_cs.c +++ b/drivers/net/ethernet/3com/3c589_cs.c @@ -685,7 +685,7 @@ static irqreturn_t el3_interrupt(int irq, void *dev_id) static void media_check(struct timer_list *t) { - struct el3_private *lp = from_timer(lp, t, media); + struct el3_private *lp = timer_container_of(lp, t, media); struct net_device *dev = lp->p_dev->priv; unsigned int ioaddr = dev->base_addr; u16 media, errs; diff --git a/drivers/net/ethernet/3com/3c59x.c b/drivers/net/ethernet/3com/3c59x.c index 1a10f5dbc4d7..8c9cc97efd4e 100644 --- a/drivers/net/ethernet/3com/3c59x.c +++ b/drivers/net/ethernet/3com/3c59x.c @@ -1783,7 +1783,7 @@ out: static void vortex_timer(struct timer_list *t) { - struct vortex_private *vp = from_timer(vp, t, timer); + struct vortex_private *vp = timer_container_of(vp, t, timer); struct net_device *dev = vp->mii.dev; void __iomem *ioaddr = vp->ioaddr; int next_tick = 60*HZ; diff --git a/drivers/net/ethernet/8390/axnet_cs.c b/drivers/net/ethernet/8390/axnet_cs.c index e5be5044e1d4..7c8213011b5c 100644 --- a/drivers/net/ethernet/8390/axnet_cs.c +++ b/drivers/net/ethernet/8390/axnet_cs.c @@ -550,7 +550,7 @@ static irqreturn_t ei_irq_wrapper(int irq, void *dev_id) static void ei_watchdog(struct timer_list *t) { - struct axnet_dev *info = from_timer(info, t, watchdog); + struct axnet_dev *info = timer_container_of(info, t, watchdog); struct net_device *dev = info->p_dev->priv; unsigned int nic_base = dev->base_addr; unsigned int mii_addr = nic_base + AXNET_MII_EEP; diff --git a/drivers/net/ethernet/8390/pcnet_cs.c b/drivers/net/ethernet/8390/pcnet_cs.c index a326f25dda09..19f9c5db3f3b 100644 --- a/drivers/net/ethernet/8390/pcnet_cs.c +++ b/drivers/net/ethernet/8390/pcnet_cs.c @@ -1018,7 +1018,7 @@ static irqreturn_t ei_irq_wrapper(int irq, void *dev_id) static void ei_watchdog(struct timer_list *t) { - struct pcnet_dev *info = from_timer(info, t, watchdog); + struct pcnet_dev *info = timer_container_of(info, t, watchdog); struct net_device *dev = info->p_dev->priv; unsigned int nic_base = dev->base_addr; unsigned int mii_addr = nic_base + DLINK_GPIO; diff --git a/drivers/net/ethernet/agere/et131x.c b/drivers/net/ethernet/agere/et131x.c index b398adacda91..678eddb36172 100644 --- a/drivers/net/ethernet/agere/et131x.c +++ b/drivers/net/ethernet/agere/et131x.c @@ -3076,7 +3076,8 @@ err_out: */ static void et131x_error_timer_handler(struct timer_list *t) { - struct et131x_adapter *adapter = from_timer(adapter, t, error_timer); + struct et131x_adapter *adapter = timer_container_of(adapter, t, + error_timer); struct phy_device *phydev = adapter->netdev->phydev; if (et1310_in_phy_coma(adapter)) { diff --git a/drivers/net/ethernet/airoha/airoha_regs.h b/drivers/net/ethernet/airoha/airoha_regs.h index 04187eb40ec6..150c85995cc1 100644 --- a/drivers/net/ethernet/airoha/airoha_regs.h +++ b/drivers/net/ethernet/airoha/airoha_regs.h @@ -614,8 +614,9 @@ RX19_DONE_INT_MASK | RX18_DONE_INT_MASK | \ RX17_DONE_INT_MASK | RX16_DONE_INT_MASK) -#define RX_DONE_INT_MASK (RX_DONE_HIGH_INT_MASK | RX_DONE_LOW_INT_MASK) #define RX_DONE_HIGH_OFFSET fls(RX_DONE_HIGH_INT_MASK) +#define RX_DONE_INT_MASK \ + ((RX_DONE_HIGH_INT_MASK << RX_DONE_HIGH_OFFSET) | RX_DONE_LOW_INT_MASK) #define INT_RX2_MASK(_n) \ ((RX_NO_CPU_DSCP_HIGH_INT_MASK & (_n)) | \ diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 7d9ee9867a40..86fd08f375df 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -3667,7 +3667,8 @@ static void ena_update_host_info(struct ena_admin_host_info *host_info, static void ena_timer_service(struct timer_list *t) { - struct ena_adapter *adapter = from_timer(adapter, t, timer_service); + struct ena_adapter *adapter = timer_container_of(adapter, t, + timer_service); u8 *debug_area = adapter->ena_dev->host_attr.debug_area_virt_addr; struct ena_admin_host_info *host_info = adapter->ena_dev->host_attr.host_info; diff --git a/drivers/net/ethernet/amd/a2065.c b/drivers/net/ethernet/amd/a2065.c index b923ad9e1581..ce9445425045 100644 --- a/drivers/net/ethernet/amd/a2065.c +++ b/drivers/net/ethernet/amd/a2065.c @@ -636,7 +636,7 @@ static void lance_set_multicast(struct net_device *dev) static void lance_set_multicast_retry(struct timer_list *t) { - struct lance_private *lp = from_timer(lp, t, multicast_timer); + struct lance_private *lp = timer_container_of(lp, t, multicast_timer); lance_set_multicast(lp->dev); } diff --git a/drivers/net/ethernet/amd/amd8111e.c b/drivers/net/ethernet/amd/amd8111e.c index 86522e8574cb..76e8c13d5985 100644 --- a/drivers/net/ethernet/amd/amd8111e.c +++ b/drivers/net/ethernet/amd/amd8111e.c @@ -1641,7 +1641,8 @@ static int __maybe_unused amd8111e_resume(struct device *dev_d) static void amd8111e_config_ipg(struct timer_list *t) { - struct amd8111e_priv *lp = from_timer(lp, t, ipg_data.ipg_timer); + struct amd8111e_priv *lp = timer_container_of(lp, t, + ipg_data.ipg_timer); struct ipg_info *ipg_data = &lp->ipg_data; void __iomem *mmio = lp->mmio; unsigned int prev_col_cnt = ipg_data->col_cnt; diff --git a/drivers/net/ethernet/amd/declance.c b/drivers/net/ethernet/amd/declance.c index b072ca5930fc..8d05a0c5f2d5 100644 --- a/drivers/net/ethernet/amd/declance.c +++ b/drivers/net/ethernet/amd/declance.c @@ -1004,7 +1004,7 @@ static void lance_set_multicast(struct net_device *dev) static void lance_set_multicast_retry(struct timer_list *t) { - struct lance_private *lp = from_timer(lp, t, multicast_timer); + struct lance_private *lp = timer_container_of(lp, t, multicast_timer); struct net_device *dev = lp->dev; lance_set_multicast(dev); diff --git a/drivers/net/ethernet/amd/pcnet32.c b/drivers/net/ethernet/amd/pcnet32.c index e5adafecc686..9eaefa0f5e80 100644 --- a/drivers/net/ethernet/amd/pcnet32.c +++ b/drivers/net/ethernet/amd/pcnet32.c @@ -2905,7 +2905,7 @@ static void pcnet32_check_media(struct net_device *dev, int verbose) static void pcnet32_watchdog(struct timer_list *t) { - struct pcnet32_private *lp = from_timer(lp, t, watchdog_timer); + struct pcnet32_private *lp = timer_container_of(lp, t, watchdog_timer); struct net_device *dev = lp->dev; unsigned long flags; diff --git a/drivers/net/ethernet/amd/pds_core/main.c b/drivers/net/ethernet/amd/pds_core/main.c index 4843f9249a31..9b81e1c260c2 100644 --- a/drivers/net/ethernet/amd/pds_core/main.c +++ b/drivers/net/ethernet/amd/pds_core/main.c @@ -23,7 +23,7 @@ MODULE_DEVICE_TABLE(pci, pdsc_id_table); static void pdsc_wdtimer_cb(struct timer_list *t) { - struct pdsc *pdsc = from_timer(pdsc, t, wdtimer); + struct pdsc *pdsc = timer_container_of(pdsc, t, wdtimer); dev_dbg(pdsc->dev, "%s: jiffies %ld\n", __func__, jiffies); mod_timer(&pdsc->wdtimer, diff --git a/drivers/net/ethernet/amd/sunlance.c b/drivers/net/ethernet/amd/sunlance.c index 3cd31855a5f6..0b273327f5a6 100644 --- a/drivers/net/ethernet/amd/sunlance.c +++ b/drivers/net/ethernet/amd/sunlance.c @@ -1246,7 +1246,7 @@ static void lance_set_multicast(struct net_device *dev) static void lance_set_multicast_retry(struct timer_list *t) { - struct lance_private *lp = from_timer(lp, t, multicast_timer); + struct lance_private *lp = timer_container_of(lp, t, multicast_timer); struct net_device *dev = lp->dev; lance_set_multicast(dev); diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c index 76c328721fcd..65447f9a0a59 100644 --- a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c +++ b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c @@ -534,7 +534,8 @@ static irqreturn_t xgbe_dma_isr(int irq, void *data) static void xgbe_tx_timer(struct timer_list *t) { - struct xgbe_channel *channel = from_timer(channel, t, tx_timer); + struct xgbe_channel *channel = timer_container_of(channel, t, + tx_timer); struct xgbe_prv_data *pdata = channel->pdata; struct napi_struct *napi; @@ -572,7 +573,8 @@ static void xgbe_service(struct work_struct *work) static void xgbe_service_timer(struct timer_list *t) { - struct xgbe_prv_data *pdata = from_timer(pdata, t, service_timer); + struct xgbe_prv_data *pdata = timer_container_of(pdata, t, + service_timer); struct xgbe_channel *channel; unsigned int i; diff --git a/drivers/net/ethernet/apple/bmac.c b/drivers/net/ethernet/apple/bmac.c index b50052c25a91..b3bf8d6f88e8 100644 --- a/drivers/net/ethernet/apple/bmac.c +++ b/drivers/net/ethernet/apple/bmac.c @@ -1410,7 +1410,7 @@ bmac_output(struct sk_buff *skb, struct net_device *dev) static void bmac_tx_timeout(struct timer_list *t) { - struct bmac_data *bp = from_timer(bp, t, tx_timeout); + struct bmac_data *bp = timer_container_of(bp, t, tx_timeout); struct net_device *dev = macio_get_drvdata(bp->mdev); volatile struct dbdma_regs __iomem *td = bp->tx_dma; volatile struct dbdma_regs __iomem *rd = bp->rx_dma; diff --git a/drivers/net/ethernet/apple/mace.c b/drivers/net/ethernet/apple/mace.c index 1fed112f4e68..af26905e44e3 100644 --- a/drivers/net/ethernet/apple/mace.c +++ b/drivers/net/ethernet/apple/mace.c @@ -805,7 +805,7 @@ static irqreturn_t mace_interrupt(int irq, void *dev_id) static void mace_tx_timeout(struct timer_list *t) { - struct mace_data *mp = from_timer(mp, t, tx_timeout); + struct mace_data *mp = timer_container_of(mp, t, tx_timeout); struct net_device *dev = macio_get_drvdata(mp->mdev); volatile struct mace __iomem *mb = mp->mace; volatile struct dbdma_regs __iomem *td = mp->tx_dma; diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c index e71cd10e4e1f..b24eaa5283fa 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c @@ -254,7 +254,7 @@ static void aq_nic_service_task(struct work_struct *work) static void aq_nic_service_timer_cb(struct timer_list *t) { - struct aq_nic_s *self = from_timer(self, t, service_timer); + struct aq_nic_s *self = timer_container_of(self, t, service_timer); mod_timer(&self->service_timer, jiffies + AQ_CFG_SERVICE_TIMER_INTERVAL); @@ -264,7 +264,7 @@ static void aq_nic_service_timer_cb(struct timer_list *t) static void aq_nic_polling_timer_cb(struct timer_list *t) { - struct aq_nic_s *self = from_timer(self, t, polling_timer); + struct aq_nic_s *self = timer_container_of(self, t, polling_timer); unsigned int i = 0U; for (i = 0U; self->aq_vecs > i; ++i) diff --git a/drivers/net/ethernet/atheros/ag71xx.c b/drivers/net/ethernet/atheros/ag71xx.c index 67b654889cae..d8e6f23e1432 100644 --- a/drivers/net/ethernet/atheros/ag71xx.c +++ b/drivers/net/ethernet/atheros/ag71xx.c @@ -1563,7 +1563,7 @@ err_drop: static void ag71xx_oom_timer_handler(struct timer_list *t) { - struct ag71xx *ag = from_timer(ag, t, oom_timer); + struct ag71xx *ag = timer_container_of(ag, t, oom_timer); napi_schedule(&ag->napi); } diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c index 82137f9deae9..ef1a51347351 100644 --- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c +++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c @@ -231,8 +231,8 @@ static u32 atl1c_wait_until_idle(struct atl1c_hw *hw, u32 modu_ctrl) */ static void atl1c_phy_config(struct timer_list *t) { - struct atl1c_adapter *adapter = from_timer(adapter, t, - phy_config_timer); + struct atl1c_adapter *adapter = timer_container_of(adapter, t, + phy_config_timer); struct atl1c_hw *hw = &adapter->hw; unsigned long flags; diff --git a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c index f664a0edbc49..40290028580b 100644 --- a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c +++ b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c @@ -115,8 +115,8 @@ static inline void atl1e_irq_reset(struct atl1e_adapter *adapter) */ static void atl1e_phy_config(struct timer_list *t) { - struct atl1e_adapter *adapter = from_timer(adapter, t, - phy_config_timer); + struct atl1e_adapter *adapter = timer_container_of(adapter, t, + phy_config_timer); struct atl1e_hw *hw = &adapter->hw; unsigned long flags; diff --git a/drivers/net/ethernet/atheros/atlx/atl1.c b/drivers/net/ethernet/atheros/atlx/atl1.c index 38cd84b7677c..cfdb546a09e7 100644 --- a/drivers/net/ethernet/atheros/atlx/atl1.c +++ b/drivers/net/ethernet/atheros/atlx/atl1.c @@ -2556,8 +2556,8 @@ static irqreturn_t atl1_intr(int irq, void *data) */ static void atl1_phy_config(struct timer_list *t) { - struct atl1_adapter *adapter = from_timer(adapter, t, - phy_config_timer); + struct atl1_adapter *adapter = timer_container_of(adapter, t, + phy_config_timer); struct atl1_hw *hw = &adapter->hw; unsigned long flags; diff --git a/drivers/net/ethernet/atheros/atlx/atl2.c b/drivers/net/ethernet/atheros/atlx/atl2.c index 88f65f8cf4d3..280e2f5f4aa5 100644 --- a/drivers/net/ethernet/atheros/atlx/atl2.c +++ b/drivers/net/ethernet/atheros/atlx/atl2.c @@ -1010,7 +1010,8 @@ static void atl2_tx_timeout(struct net_device *netdev, unsigned int txqueue) */ static void atl2_watchdog(struct timer_list *t) { - struct atl2_adapter *adapter = from_timer(adapter, t, watchdog_timer); + struct atl2_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); if (!test_bit(__ATL2_DOWN, &adapter->flags)) { u32 drop_rxd, drop_rxs; @@ -1035,8 +1036,8 @@ static void atl2_watchdog(struct timer_list *t) */ static void atl2_phy_config(struct timer_list *t) { - struct atl2_adapter *adapter = from_timer(adapter, t, - phy_config_timer); + struct atl2_adapter *adapter = timer_container_of(adapter, t, + phy_config_timer); struct atl2_hw *hw = &adapter->hw; unsigned long flags; diff --git a/drivers/net/ethernet/broadcom/b44.c b/drivers/net/ethernet/broadcom/b44.c index c91884373429..8267417b3750 100644 --- a/drivers/net/ethernet/broadcom/b44.c +++ b/drivers/net/ethernet/broadcom/b44.c @@ -575,7 +575,7 @@ static void b44_check_phy(struct b44 *bp) static void b44_timer(struct timer_list *t) { - struct b44 *bp = from_timer(bp, t, timer); + struct b44 *bp = timer_container_of(bp, t, timer); spin_lock_irq(&bp->lock); diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 19611bdd86e6..92204fea1f08 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -286,7 +286,7 @@ static int bcm_enet_refill_rx(struct net_device *dev, bool napi_mode) */ static void bcm_enet_refill_rx_timer(struct timer_list *t) { - struct bcm_enet_priv *priv = from_timer(priv, t, rx_timeout); + struct bcm_enet_priv *priv = timer_container_of(priv, t, rx_timeout); struct net_device *dev = priv->net_dev; spin_lock(&priv->rx_lock); @@ -2001,7 +2001,7 @@ static inline int bcm_enet_port_is_rgmii(int portid) */ static void swphy_poll_timer(struct timer_list *t) { - struct bcm_enet_priv *priv = from_timer(priv, t, swphy_poll); + struct bcm_enet_priv *priv = timer_container_of(priv, t, swphy_poll); unsigned int i; for (i = 0; i < priv->num_ports; i++) { diff --git a/drivers/net/ethernet/broadcom/bnx2.c b/drivers/net/ethernet/broadcom/bnx2.c index ec0c9584f3bb..cb1011f6fd30 100644 --- a/drivers/net/ethernet/broadcom/bnx2.c +++ b/drivers/net/ethernet/broadcom/bnx2.c @@ -6163,7 +6163,7 @@ bnx2_5708_serdes_timer(struct bnx2 *bp) static void bnx2_timer(struct timer_list *t) { - struct bnx2 *bp = from_timer(bp, t, timer); + struct bnx2 *bp = timer_container_of(bp, t, timer); if (!netif_running(bp->dev)) return; diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index f522ca8ff66b..c9a1a1d504c0 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -5783,7 +5783,7 @@ void bnx2x_drv_pulse(struct bnx2x *bp) static void bnx2x_timer(struct timer_list *t) { - struct bnx2x *bp = from_timer(bp, t, timer); + struct bnx2x *bp = timer_container_of(bp, t, timer); if (!netif_running(bp->dev)) return; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index d5495762c945..869580b6f70d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -13995,7 +13995,7 @@ fw_reset: static void bnxt_timer(struct timer_list *t) { - struct bnxt *bp = from_timer(bp, t, timer); + struct bnxt *bp = timer_container_of(bp, t, timer); struct net_device *dev = bp->dev; if (!netif_running(dev) || !test_bit(BNXT_STATE_OPEN, &bp->state)) diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c index ff47e96b9124..91104cc2c238 100644 --- a/drivers/net/ethernet/broadcom/tg3.c +++ b/drivers/net/ethernet/broadcom/tg3.c @@ -11062,7 +11062,7 @@ static void tg3_chk_missed_msi(struct tg3 *tp) static void tg3_timer(struct timer_list *t) { - struct tg3 *tp = from_timer(tp, t, timer); + struct tg3 *tp = timer_container_of(tp, t, timer); spin_lock(&tp->lock); diff --git a/drivers/net/ethernet/brocade/bna/bnad.c b/drivers/net/ethernet/brocade/bna/bnad.c index 50eb54ecf1ba..9bed33295839 100644 --- a/drivers/net/ethernet/brocade/bna/bnad.c +++ b/drivers/net/ethernet/brocade/bna/bnad.c @@ -1688,7 +1688,8 @@ err_return: static void bnad_ioc_timeout(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, bna.ioceth.ioc.ioc_timer); + struct bnad *bnad = timer_container_of(bnad, t, + bna.ioceth.ioc.ioc_timer); unsigned long flags; spin_lock_irqsave(&bnad->bna_lock, flags); @@ -1699,7 +1700,8 @@ bnad_ioc_timeout(struct timer_list *t) static void bnad_ioc_hb_check(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, bna.ioceth.ioc.hb_timer); + struct bnad *bnad = timer_container_of(bnad, t, + bna.ioceth.ioc.hb_timer); unsigned long flags; spin_lock_irqsave(&bnad->bna_lock, flags); @@ -1710,7 +1712,8 @@ bnad_ioc_hb_check(struct timer_list *t) static void bnad_iocpf_timeout(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, bna.ioceth.ioc.iocpf_timer); + struct bnad *bnad = timer_container_of(bnad, t, + bna.ioceth.ioc.iocpf_timer); unsigned long flags; spin_lock_irqsave(&bnad->bna_lock, flags); @@ -1721,7 +1724,8 @@ bnad_iocpf_timeout(struct timer_list *t) static void bnad_iocpf_sem_timeout(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, bna.ioceth.ioc.sem_timer); + struct bnad *bnad = timer_container_of(bnad, t, + bna.ioceth.ioc.sem_timer); unsigned long flags; spin_lock_irqsave(&bnad->bna_lock, flags); @@ -1743,7 +1747,7 @@ bnad_iocpf_sem_timeout(struct timer_list *t) static void bnad_dim_timeout(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, dim_timer); + struct bnad *bnad = timer_container_of(bnad, t, dim_timer); struct bnad_rx_info *rx_info; struct bnad_rx_ctrl *rx_ctrl; int i, j; @@ -1776,7 +1780,7 @@ bnad_dim_timeout(struct timer_list *t) static void bnad_stats_timeout(struct timer_list *t) { - struct bnad *bnad = from_timer(bnad, t, stats_timer); + struct bnad *bnad = timer_container_of(bnad, t, stats_timer); unsigned long flags; if (!netif_running(bnad->netdev) || diff --git a/drivers/net/ethernet/chelsio/cxgb/sge.c b/drivers/net/ethernet/chelsio/cxgb/sge.c index a10923c7e25c..5f354cf62cdd 100644 --- a/drivers/net/ethernet/chelsio/cxgb/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb/sge.c @@ -1922,7 +1922,7 @@ send: static void sge_tx_reclaim_cb(struct timer_list *t) { int i; - struct sge *sge = from_timer(sge, t, tx_reclaim_timer); + struct sge *sge = timer_container_of(sge, t, tx_reclaim_timer); for (i = 0; i < SGE_CMDQ_N; ++i) { struct cmdQ *q = &sge->cmdQ[i]; @@ -2017,7 +2017,7 @@ void t1_sge_start(struct sge *sge) */ static void espibug_workaround_t204(struct timer_list *t) { - struct sge *sge = from_timer(sge, t, espibug_timer); + struct sge *sge = timer_container_of(sge, t, espibug_timer); struct adapter *adapter = sge->adapter; unsigned int nports = adapter->params.nports; u32 seop[MAX_NPORTS]; @@ -2060,7 +2060,7 @@ static void espibug_workaround_t204(struct timer_list *t) static void espibug_workaround(struct timer_list *t) { - struct sge *sge = from_timer(sge, t, espibug_timer); + struct sge *sge = timer_container_of(sge, t, espibug_timer); struct adapter *adapter = sge->adapter; if (netif_running(adapter->port[0].dev)) { diff --git a/drivers/net/ethernet/chelsio/cxgb3/sge.c b/drivers/net/ethernet/chelsio/cxgb3/sge.c index bd5c3b3fa5e3..b59735d0e065 100644 --- a/drivers/net/ethernet/chelsio/cxgb3/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb3/sge.c @@ -2906,7 +2906,7 @@ void t3_sge_err_intr_handler(struct adapter *adapter) */ static void sge_timer_tx(struct timer_list *t) { - struct sge_qset *qs = from_timer(qs, t, tx_reclaim_timer); + struct sge_qset *qs = timer_container_of(qs, t, tx_reclaim_timer); struct port_info *pi = netdev_priv(qs->netdev); struct adapter *adap = pi->adapter; unsigned int tbd[SGE_TXQ_PER_SET] = {0, 0}; @@ -2947,7 +2947,7 @@ static void sge_timer_tx(struct timer_list *t) static void sge_timer_rx(struct timer_list *t) { spinlock_t *lock; - struct sge_qset *qs = from_timer(qs, t, rx_reclaim_timer); + struct sge_qset *qs = timer_container_of(qs, t, rx_reclaim_timer); struct port_info *pi = netdev_priv(qs->netdev); struct adapter *adap = pi->adapter; u32 status; diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c index 69d045d769c4..0765d000eaef 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c @@ -1051,7 +1051,7 @@ static void ch_flower_stats_handler(struct work_struct *work) static void ch_flower_stats_cb(struct timer_list *t) { - struct adapter *adap = from_timer(adap, t, flower_stats_timer); + struct adapter *adap = timer_container_of(adap, t, flower_stats_timer); schedule_work(&adap->flower_stats_work); } diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index f2d533acb056..64402e3646b3 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -4234,7 +4234,7 @@ static void sge_rx_timer_cb(struct timer_list *t) { unsigned long m; unsigned int i; - struct adapter *adap = from_timer(adap, t, sge.rx_timer); + struct adapter *adap = timer_container_of(adap, t, sge.rx_timer); struct sge *s = &adap->sge; for (i = 0; i < BITS_TO_LONGS(s->egr_sz); i++) @@ -4269,7 +4269,7 @@ done: static void sge_tx_timer_cb(struct timer_list *t) { - struct adapter *adap = from_timer(adap, t, sge.tx_timer); + struct adapter *adap = timer_container_of(adap, t, sge.tx_timer); struct sge *s = &adap->sge; unsigned long m, period; unsigned int i, budget; diff --git a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c index f42af01f4114..4e6ecb9c8dcc 100644 --- a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c @@ -2062,7 +2062,7 @@ irq_handler_t t4vf_intr_handler(struct adapter *adapter) */ static void sge_rx_timer_cb(struct timer_list *t) { - struct adapter *adapter = from_timer(adapter, t, sge.rx_timer); + struct adapter *adapter = timer_container_of(adapter, t, sge.rx_timer); struct sge *s = &adapter->sge; unsigned int i; @@ -2121,7 +2121,7 @@ static void sge_rx_timer_cb(struct timer_list *t) */ static void sge_tx_timer_cb(struct timer_list *t) { - struct adapter *adapter = from_timer(adapter, t, sge.tx_timer); + struct adapter *adapter = timer_container_of(adapter, t, sge.tx_timer); struct sge *s = &adapter->sge; unsigned int i, budget; diff --git a/drivers/net/ethernet/cisco/enic/enic_clsf.c b/drivers/net/ethernet/cisco/enic/enic_clsf.c index 9900993b6aea..837f954873ee 100644 --- a/drivers/net/ethernet/cisco/enic/enic_clsf.c +++ b/drivers/net/ethernet/cisco/enic/enic_clsf.c @@ -125,7 +125,7 @@ struct enic_rfs_fltr_node *htbl_fltr_search(struct enic *enic, u16 fltr_id) #ifdef CONFIG_RFS_ACCEL void enic_flow_may_expire(struct timer_list *t) { - struct enic *enic = from_timer(enic, t, rfs_h.rfs_may_expire); + struct enic *enic = timer_container_of(enic, t, rfs_h.rfs_may_expire); bool res; int j; diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c index 6ef8a0d90bce..773f5ad972a2 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -1510,7 +1510,7 @@ static int enic_poll_msix_rq(struct napi_struct *napi, int budget) static void enic_notify_timer(struct timer_list *t) { - struct enic *enic = from_timer(enic, t, notify_timer); + struct enic *enic = timer_container_of(enic, t, notify_timer); enic_notify_check(enic); diff --git a/drivers/net/ethernet/dec/tulip/de2104x.c b/drivers/net/ethernet/dec/tulip/de2104x.c index f9339d0772b5..f9504f340c4a 100644 --- a/drivers/net/ethernet/dec/tulip/de2104x.c +++ b/drivers/net/ethernet/dec/tulip/de2104x.c @@ -963,7 +963,7 @@ static void de_next_media (struct de_private *de, const u32 *media, static void de21040_media_timer (struct timer_list *t) { - struct de_private *de = from_timer(de, t, media_timer); + struct de_private *de = timer_container_of(de, t, media_timer); struct net_device *dev = de->dev; u32 status = dr32(SIAStatus); unsigned int carrier; @@ -1044,7 +1044,7 @@ static unsigned int de_ok_to_advertise (struct de_private *de, u32 new_media) static void de21041_media_timer (struct timer_list *t) { - struct de_private *de = from_timer(de, t, media_timer); + struct de_private *de = timer_container_of(de, t, media_timer); struct net_device *dev = de->dev; u32 status = dr32(SIAStatus); unsigned int carrier; diff --git a/drivers/net/ethernet/dec/tulip/dmfe.c b/drivers/net/ethernet/dec/tulip/dmfe.c index ae34b95ed676..2d3bd343b6e6 100644 --- a/drivers/net/ethernet/dec/tulip/dmfe.c +++ b/drivers/net/ethernet/dec/tulip/dmfe.c @@ -1115,7 +1115,7 @@ static const struct ethtool_ops netdev_ethtool_ops = { static void dmfe_timer(struct timer_list *t) { - struct dmfe_board_info *db = from_timer(db, t, timer); + struct dmfe_board_info *db = timer_container_of(db, t, timer); struct net_device *dev = pci_get_drvdata(db->pdev); void __iomem *ioaddr = db->ioaddr; u32 tmp_cr8; diff --git a/drivers/net/ethernet/dec/tulip/interrupt.c b/drivers/net/ethernet/dec/tulip/interrupt.c index 2d926a26fbb9..0a12cb9b3ba7 100644 --- a/drivers/net/ethernet/dec/tulip/interrupt.c +++ b/drivers/net/ethernet/dec/tulip/interrupt.c @@ -104,7 +104,7 @@ int tulip_refill_rx(struct net_device *dev) void oom_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, oom_timer); + struct tulip_private *tp = timer_container_of(tp, t, oom_timer); napi_schedule(&tp->napi); } diff --git a/drivers/net/ethernet/dec/tulip/pnic.c b/drivers/net/ethernet/dec/tulip/pnic.c index 653bde48ef44..1de5ed967070 100644 --- a/drivers/net/ethernet/dec/tulip/pnic.c +++ b/drivers/net/ethernet/dec/tulip/pnic.c @@ -86,7 +86,7 @@ void pnic_lnk_change(struct net_device *dev, int csr5) void pnic_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, timer); + struct tulip_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; void __iomem *ioaddr = tp->base_addr; int next_tick = 60*HZ; diff --git a/drivers/net/ethernet/dec/tulip/pnic2.c b/drivers/net/ethernet/dec/tulip/pnic2.c index 2e3bdc0fcdc0..39c410bf224e 100644 --- a/drivers/net/ethernet/dec/tulip/pnic2.c +++ b/drivers/net/ethernet/dec/tulip/pnic2.c @@ -78,7 +78,7 @@ void pnic2_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, timer); + struct tulip_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; void __iomem *ioaddr = tp->base_addr; int next_tick = 60*HZ; diff --git a/drivers/net/ethernet/dec/tulip/timer.c b/drivers/net/ethernet/dec/tulip/timer.c index 642e9dfc5451..ca0c509b601c 100644 --- a/drivers/net/ethernet/dec/tulip/timer.c +++ b/drivers/net/ethernet/dec/tulip/timer.c @@ -139,7 +139,7 @@ void tulip_media_task(struct work_struct *work) void mxic_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, timer); + struct tulip_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; void __iomem *ioaddr = tp->base_addr; int next_tick = 60*HZ; @@ -156,7 +156,7 @@ void mxic_timer(struct timer_list *t) void comet_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, timer); + struct tulip_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; int next_tick = 2*HZ; diff --git a/drivers/net/ethernet/dec/tulip/tulip_core.c b/drivers/net/ethernet/dec/tulip/tulip_core.c index bec76e7bf5dd..5b7e6eb080f3 100644 --- a/drivers/net/ethernet/dec/tulip/tulip_core.c +++ b/drivers/net/ethernet/dec/tulip/tulip_core.c @@ -114,7 +114,7 @@ int tulip_debug = 1; static void tulip_timer(struct timer_list *t) { - struct tulip_private *tp = from_timer(tp, t, timer); + struct tulip_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; if (netif_running(dev)) diff --git a/drivers/net/ethernet/dec/tulip/uli526x.c b/drivers/net/ethernet/dec/tulip/uli526x.c index 3f1bd670700b..6e4d8d31aba9 100644 --- a/drivers/net/ethernet/dec/tulip/uli526x.c +++ b/drivers/net/ethernet/dec/tulip/uli526x.c @@ -1014,7 +1014,7 @@ static const struct ethtool_ops netdev_ethtool_ops = { static void uli526x_timer(struct timer_list *t) { - struct uli526x_board_info *db = from_timer(db, t, timer); + struct uli526x_board_info *db = timer_container_of(db, t, timer); struct net_device *dev = pci_get_drvdata(db->pdev); struct uli_phy_ops *phy = &db->phy; void __iomem *ioaddr = db->ioaddr; diff --git a/drivers/net/ethernet/dec/tulip/winbond-840.c b/drivers/net/ethernet/dec/tulip/winbond-840.c index e593273b2867..a24a25a5f73d 100644 --- a/drivers/net/ethernet/dec/tulip/winbond-840.c +++ b/drivers/net/ethernet/dec/tulip/winbond-840.c @@ -763,7 +763,7 @@ static inline void update_csr6(struct net_device *dev, int new) static void netdev_timer(struct timer_list *t) { - struct netdev_private *np = from_timer(np, t, timer); + struct netdev_private *np = timer_container_of(np, t, timer); struct net_device *dev = pci_get_drvdata(np->pci_dev); void __iomem *ioaddr = np->base_addr; diff --git a/drivers/net/ethernet/dlink/dl2k.c b/drivers/net/ethernet/dlink/dl2k.c index 038a0400c1f9..da9b7715df05 100644 --- a/drivers/net/ethernet/dlink/dl2k.c +++ b/drivers/net/ethernet/dlink/dl2k.c @@ -650,7 +650,7 @@ static int rio_open(struct net_device *dev) static void rio_timer (struct timer_list *t) { - struct netdev_private *np = from_timer(np, t, timer); + struct netdev_private *np = timer_container_of(np, t, timer); struct net_device *dev = pci_get_drvdata(np->pdev); unsigned int entry; int next_tick = 1*HZ; diff --git a/drivers/net/ethernet/fealnx.c b/drivers/net/ethernet/fealnx.c index 670b68201376..6ac8547ef9b8 100644 --- a/drivers/net/ethernet/fealnx.c +++ b/drivers/net/ethernet/fealnx.c @@ -1074,7 +1074,7 @@ static void allocate_rx_buffers(struct net_device *dev) static void netdev_timer(struct timer_list *t) { - struct netdev_private *np = from_timer(np, t, timer); + struct netdev_private *np = timer_container_of(np, t, timer); struct net_device *dev = np->mii.dev; void __iomem *ioaddr = np->mem; int old_crvalue = np->crvalue; @@ -1163,7 +1163,7 @@ static void enable_rxtx(struct net_device *dev) static void reset_timer(struct timer_list *t) { - struct netdev_private *np = from_timer(np, t, reset_timer); + struct netdev_private *np = timer_container_of(np, t, reset_timer); struct net_device *dev = np->mii.dev; unsigned long flags; diff --git a/drivers/net/ethernet/freescale/enetc/Kconfig b/drivers/net/ethernet/freescale/enetc/Kconfig index e917132d3714..54b0f0a5a6bb 100644 --- a/drivers/net/ethernet/freescale/enetc/Kconfig +++ b/drivers/net/ethernet/freescale/enetc/Kconfig @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 config FSL_ENETC_CORE tristate + select NXP_NETC_LIB if NXP_NTMP help This module supports common functionality between the PF and VF drivers for the NXP ENETC controller. @@ -22,6 +23,9 @@ config NXP_NETC_LIB Switch, such as NETC Table Management Protocol (NTMP) 2.0, common tc flower and debugfs interfaces and so on. +config NXP_NTMP + bool + config FSL_ENETC tristate "ENETC PF driver" depends on PCI_MSI @@ -45,7 +49,7 @@ config NXP_ENETC4 select FSL_ENETC_CORE select FSL_ENETC_MDIO select NXP_ENETC_PF_COMMON - select NXP_NETC_LIB + select NXP_NTMP select PHYLINK select DIMLIB help diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 7cd1eda0b449..dc35a23ec47f 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -268,7 +268,8 @@ static void gve_stats_report_schedule(struct gve_priv *priv) static void gve_stats_report_timer(struct timer_list *t) { - struct gve_priv *priv = from_timer(priv, t, stats_report_timer); + struct gve_priv *priv = timer_container_of(priv, t, + stats_report_timer); mod_timer(&priv->stats_report_timer, round_jiffies(jiffies + diff --git a/drivers/net/ethernet/hisilicon/hns/hns_enet.c b/drivers/net/ethernet/hisilicon/hns/hns_enet.c index d98f8d3ce7c8..e905f10b894e 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_enet.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_enet.c @@ -2075,7 +2075,7 @@ static void hns_nic_task_schedule(struct hns_nic_priv *priv) static void hns_nic_service_timer(struct timer_list *t) { - struct hns_nic_priv *priv = from_timer(priv, t, service_timer); + struct hns_nic_priv *priv = timer_container_of(priv, t, service_timer); (void)mod_timer(&priv->service_timer, jiffies + SERVICE_TIMER_HZ); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c index 3e28a08934ab..a7de67699a01 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c @@ -4503,7 +4503,7 @@ static void hclge_set_def_reset_request(struct hnae3_ae_dev *ae_dev, static void hclge_reset_timer(struct timer_list *t) { - struct hclge_dev *hdev = from_timer(hdev, t, reset_timer); + struct hclge_dev *hdev = timer_container_of(hdev, t, reset_timer); /* if default_reset_request has no value, it means that this reset * request has already be handled, so just return here diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index dada42e7e0ec..c4f35e8e2177 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -2057,7 +2057,7 @@ static enum hclgevf_evt_cause hclgevf_check_evt_cause(struct hclgevf_dev *hdev, static void hclgevf_reset_timer(struct timer_list *t) { - struct hclgevf_dev *hdev = from_timer(hdev, t, reset_timer); + struct hclgevf_dev *hdev = timer_container_of(hdev, t, reset_timer); hclgevf_clear_event_cause(hdev, HCLGEVF_VECTOR0_EVENT_RST); hclgevf_reset_task_schedule(hdev); diff --git a/drivers/net/ethernet/intel/e100.c b/drivers/net/ethernet/intel/e100.c index c0ead54ea186..5c56c1edd492 100644 --- a/drivers/net/ethernet/intel/e100.c +++ b/drivers/net/ethernet/intel/e100.c @@ -1682,7 +1682,7 @@ static void e100_adjust_adaptive_ifs(struct nic *nic, int speed, int duplex) static void e100_watchdog(struct timer_list *t) { - struct nic *nic = from_timer(nic, t, watchdog); + struct nic *nic = timer_container_of(nic, t, watchdog); struct ethtool_cmd cmd = { .cmd = ETHTOOL_GSET }; u32 speed; diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c index 3f089c3d47b2..d8595e84326d 100644 --- a/drivers/net/ethernet/intel/e1000/e1000_main.c +++ b/drivers/net/ethernet/intel/e1000/e1000_main.c @@ -477,10 +477,6 @@ static void e1000_down_and_stop(struct e1000_adapter *adapter) cancel_delayed_work_sync(&adapter->phy_info_task); cancel_delayed_work_sync(&adapter->fifo_stall_task); - - /* Only kill reset task if adapter is not resetting */ - if (!test_bit(__E1000_RESETTING, &adapter->flags)) - cancel_work_sync(&adapter->reset_task); } void e1000_down(struct e1000_adapter *adapter) @@ -1266,6 +1262,10 @@ static void e1000_remove(struct pci_dev *pdev) unregister_netdev(netdev); + /* Only kill reset task if adapter is not resetting */ + if (!test_bit(__E1000_RESETTING, &adapter->flags)) + cancel_work_sync(&adapter->reset_task); + e1000_phy_hw_reset(hw); kfree(adapter->tx_ring); diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c index e0f492a6723f..a96f4cfa6e17 100644 --- a/drivers/net/ethernet/intel/e1000e/netdev.c +++ b/drivers/net/ethernet/intel/e1000e/netdev.c @@ -4853,7 +4853,8 @@ static void e1000e_update_phy_task(struct work_struct *work) **/ static void e1000_update_phy_info(struct timer_list *t) { - struct e1000_adapter *adapter = from_timer(adapter, t, phy_info_timer); + struct e1000_adapter *adapter = timer_container_of(adapter, t, + phy_info_timer); if (test_bit(__E1000_DOWN, &adapter->state)) return; @@ -5189,7 +5190,8 @@ static void e1000e_check_82574_phy_workaround(struct e1000_adapter *adapter) **/ static void e1000_watchdog(struct timer_list *t) { - struct e1000_adapter *adapter = from_timer(adapter, t, watchdog_timer); + struct e1000_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); /* Do the rest outside of interrupt context */ schedule_work(&adapter->watchdog_task); diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_pci.c b/drivers/net/ethernet/intel/fm10k/fm10k_pci.c index 21267ab603ef..ae5fe34659cf 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_pci.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_pci.c @@ -199,8 +199,8 @@ static void fm10k_start_service_event(struct fm10k_intfc *interface) **/ static void fm10k_service_timer(struct timer_list *t) { - struct fm10k_intfc *interface = from_timer(interface, t, - service_timer); + struct fm10k_intfc *interface = timer_container_of(interface, t, + service_timer); /* Reset the timer */ mod_timer(&interface->service_timer, (HZ * 2) + jiffies); diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 120d68654e3f..f1c9e575703e 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -11412,7 +11412,7 @@ static void i40e_service_task(struct work_struct *work) **/ static void i40e_service_timer(struct timer_list *t) { - struct i40e_pf *pf = from_timer(pf, t, service_timer); + struct i40e_pf *pf = timer_container_of(pf, t, service_timer); mod_timer(&pf->service_timer, round_jiffies(jiffies + pf->service_timer_period)); diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c index 1120f8e4bb67..88e6bef69342 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c @@ -1546,8 +1546,8 @@ static void i40e_cleanup_reset_vf(struct i40e_vf *vf) * @vf: pointer to the VF structure * @flr: VFLR was issued or not * - * Returns true if the VF is in reset, resets successfully, or resets - * are disabled and false otherwise. + * Return: True if reset was performed successfully or if resets are disabled. + * False if reset is already in progress. **/ bool i40e_reset_vf(struct i40e_vf *vf, bool flr) { @@ -1566,7 +1566,7 @@ bool i40e_reset_vf(struct i40e_vf *vf, bool flr) /* If VF is being reset already we don't need to continue. */ if (test_and_set_bit(I40E_VF_STATE_RESETTING, &vf->vf_states)) - return true; + return false; i40e_trigger_vf_reset(vf, flr); @@ -4328,7 +4328,10 @@ int i40e_vc_process_vflr_event(struct i40e_pf *pf) reg = rd32(hw, I40E_GLGEN_VFLRSTAT(reg_idx)); if (reg & BIT(bit_idx)) /* i40e_reset_vf will clear the bit in GLGEN_VFLRSTAT */ - i40e_reset_vf(vf, true); + if (!i40e_reset_vf(vf, true)) { + /* At least one VF did not finish resetting, retry next time */ + set_bit(__I40E_VFLR_EVENT_PENDING, pf->state); + } } return 0; diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 2c0bb41809a4..81d7249d1149 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -3209,6 +3209,17 @@ static void iavf_reset_task(struct work_struct *work) } continue_reset: + /* If we are still early in the state machine, just restart. */ + if (adapter->state <= __IAVF_INIT_FAILED) { + iavf_shutdown_adminq(hw); + iavf_change_state(adapter, __IAVF_STARTUP); + iavf_startup(adapter); + queue_delayed_work(adapter->wq, &adapter->watchdog_task, + msecs_to_jiffies(30)); + netdev_unlock(netdev); + return; + } + /* We don't use netif_running() because it may be true prior to * ndo_open() returning, so we can't assume it means all our open * tasks have finished, since we're not holding the rtnl_lock here. diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c index a6f0e5990be2..07f0d0a0f1e2 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c @@ -79,6 +79,23 @@ iavf_poll_virtchnl_msg(struct iavf_hw *hw, struct iavf_arq_event_info *event, return iavf_status_to_errno(status); received_op = (enum virtchnl_ops)le32_to_cpu(event->desc.cookie_high); + + if (received_op == VIRTCHNL_OP_EVENT) { + struct iavf_adapter *adapter = hw->back; + struct virtchnl_pf_event *vpe = + (struct virtchnl_pf_event *)event->msg_buf; + + if (vpe->event != VIRTCHNL_EVENT_RESET_IMPENDING) + continue; + + dev_info(&adapter->pdev->dev, "Reset indication received from the PF\n"); + if (!(adapter->flags & IAVF_FLAG_RESET_PENDING)) + iavf_schedule_reset(adapter, + IAVF_FLAG_RESET_PENDING); + + return -EIO; + } + if (op_to_poll == received_op) break; } diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index d97d4b25b30d..0a11b4281092 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -1743,7 +1743,7 @@ static void ice_service_task_restart(struct ice_pf *pf) */ static void ice_service_timer(struct timer_list *t) { - struct ice_pf *pf = from_timer(pf, t, serv_tmr); + struct ice_pf *pf = timer_container_of(pf, t, serv_tmr); mod_timer(&pf->serv_tmr, round_jiffies(pf->serv_tmr_period + jiffies)); ice_service_task_schedule(pf); diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c index b79a148ed0f2..55cad824c5b9 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp.c @@ -2299,6 +2299,7 @@ static int ice_capture_crosststamp(ktime_t *device, ts = ((u64)ts_hi << 32) | ts_lo; system->cycles = ts; system->cs_id = CSID_X86_ART; + system->use_nsecs = true; /* Read Device source clock time */ ts_lo = rd32(hw, cfg->dev_time_l[tmr_idx]); diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c index 1cca9b2262e8..ae83c3914e29 100644 --- a/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c +++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c @@ -1450,7 +1450,8 @@ err_free_pkt: */ static void ice_vf_fdir_timer(struct timer_list *t) { - struct ice_vf_fdir_ctx *ctx_irq = from_timer(ctx_irq, t, rx_tmr); + struct ice_vf_fdir_ctx *ctx_irq = timer_container_of(ctx_irq, t, + rx_tmr); struct ice_vf_fdir_ctx *ctx_done; struct ice_vf_fdir *fdir; unsigned long flags; diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 9e9a5900e6e5..b76a154e635e 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -5465,7 +5465,8 @@ static void igb_spoof_check(struct igb_adapter *adapter) */ static void igb_update_phy_info(struct timer_list *t) { - struct igb_adapter *adapter = from_timer(adapter, t, phy_info_timer); + struct igb_adapter *adapter = timer_container_of(adapter, t, + phy_info_timer); igb_get_phy_info(&adapter->hw); } @@ -5555,7 +5556,8 @@ static void igb_check_lvmmc(struct igb_adapter *adapter) **/ static void igb_watchdog(struct timer_list *t) { - struct igb_adapter *adapter = from_timer(adapter, t, watchdog_timer); + struct igb_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); /* Do the rest outside of interrupt context */ schedule_work(&adapter->watchdog_task); } diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c index beb01248600f..e55dd9345833 100644 --- a/drivers/net/ethernet/intel/igbvf/netdev.c +++ b/drivers/net/ethernet/intel/igbvf/netdev.c @@ -1900,7 +1900,8 @@ static bool igbvf_has_link(struct igbvf_adapter *adapter) **/ static void igbvf_watchdog(struct timer_list *t) { - struct igbvf_adapter *adapter = from_timer(adapter, t, watchdog_timer); + struct igbvf_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); /* Do the rest outside of interrupt context */ schedule_work(&adapter->watchdog_task); diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 27575a1e1777..686793c539f2 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -5749,7 +5749,8 @@ static void igc_clear_interrupt_scheme(struct igc_adapter *adapter) */ static void igc_update_phy_info(struct timer_list *t) { - struct igc_adapter *adapter = from_timer(adapter, t, phy_info_timer); + struct igc_adapter *adapter = timer_container_of(adapter, t, + phy_info_timer); igc_get_phy_info(&adapter->hw); } @@ -5791,7 +5792,8 @@ bool igc_has_link(struct igc_adapter *adapter) */ static void igc_watchdog(struct timer_list *t) { - struct igc_adapter *adapter = from_timer(adapter, t, watchdog_timer); + struct igc_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); /* Do the rest outside of interrupt context */ schedule_work(&adapter->watchdog_task); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 03d31e5b131d..cba860f0e1f1 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -8310,7 +8310,8 @@ static void ixgbe_sfp_link_config_subtask(struct ixgbe_adapter *adapter) **/ static void ixgbe_service_timer(struct timer_list *t) { - struct ixgbe_adapter *adapter = from_timer(adapter, t, service_timer); + struct ixgbe_adapter *adapter = timer_container_of(adapter, t, + service_timer); unsigned long next_event_offset; /* poll faster when waiting for link */ diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index a217c5c04804..535d0f71f521 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -3176,8 +3176,8 @@ void ixgbevf_update_stats(struct ixgbevf_adapter *adapter) **/ static void ixgbevf_service_timer(struct timer_list *t) { - struct ixgbevf_adapter *adapter = from_timer(adapter, t, - service_timer); + struct ixgbevf_adapter *adapter = timer_container_of(adapter, t, + service_timer); /* Reset the timer */ mod_timer(&adapter->service_timer, (HZ * 2) + jiffies); diff --git a/drivers/net/ethernet/korina.c b/drivers/net/ethernet/korina.c index 1e2ac1a5f099..891a94d89f4b 100644 --- a/drivers/net/ethernet/korina.c +++ b/drivers/net/ethernet/korina.c @@ -900,7 +900,8 @@ static void korina_check_media(struct net_device *dev, unsigned int init_media) static void korina_poll_media(struct timer_list *t) { - struct korina_private *lp = from_timer(lp, t, media_check_timer); + struct korina_private *lp = timer_container_of(lp, t, + media_check_timer); struct net_device *dev = lp->dev; korina_check_media(dev, 0); diff --git a/drivers/net/ethernet/marvell/mv643xx_eth.c b/drivers/net/ethernet/marvell/mv643xx_eth.c index 8cc888bf6094..0ab52c57c648 100644 --- a/drivers/net/ethernet/marvell/mv643xx_eth.c +++ b/drivers/net/ethernet/marvell/mv643xx_eth.c @@ -1333,7 +1333,8 @@ static void mib_counters_update(struct mv643xx_eth_private *mp) static void mib_counters_timer_wrapper(struct timer_list *t) { - struct mv643xx_eth_private *mp = from_timer(mp, t, mib_counters_timer); + struct mv643xx_eth_private *mp = timer_container_of(mp, t, + mib_counters_timer); mib_counters_update(mp); mod_timer(&mp->mib_counters_timer, jiffies + 30 * HZ); } @@ -2306,7 +2307,7 @@ static int mv643xx_eth_poll(struct napi_struct *napi, int budget) static inline void oom_timer_wrapper(struct timer_list *t) { - struct mv643xx_eth_private *mp = from_timer(mp, t, rx_oom); + struct mv643xx_eth_private *mp = timer_container_of(mp, t, rx_oom); napi_schedule(&mp->napi); } diff --git a/drivers/net/ethernet/marvell/pxa168_eth.c b/drivers/net/ethernet/marvell/pxa168_eth.c index 72c1967768f4..e4cfdc8bc055 100644 --- a/drivers/net/ethernet/marvell/pxa168_eth.c +++ b/drivers/net/ethernet/marvell/pxa168_eth.c @@ -353,7 +353,7 @@ static void rxq_refill(struct net_device *dev) static inline void rxq_refill_timer_wrapper(struct timer_list *t) { - struct pxa168_eth_private *pep = from_timer(pep, t, timeout); + struct pxa168_eth_private *pep = timer_container_of(pep, t, timeout); napi_schedule(&pep->napi); } diff --git a/drivers/net/ethernet/marvell/skge.c b/drivers/net/ethernet/marvell/skge.c index b2081d6e34f0..05349a0b2db1 100644 --- a/drivers/net/ethernet/marvell/skge.c +++ b/drivers/net/ethernet/marvell/skge.c @@ -1494,7 +1494,7 @@ static int xm_check_link(struct net_device *dev) */ static void xm_link_timer(struct timer_list *t) { - struct skge_port *skge = from_timer(skge, t, link_timer); + struct skge_port *skge = timer_container_of(skge, t, link_timer); struct net_device *dev = skge->netdev; struct skge_hw *hw = skge->hw; int port = skge->port; diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c index e2a9aae8bc9b..3831f533b9db 100644 --- a/drivers/net/ethernet/marvell/sky2.c +++ b/drivers/net/ethernet/marvell/sky2.c @@ -2961,7 +2961,7 @@ static int sky2_rx_hung(struct net_device *dev) static void sky2_watchdog(struct timer_list *t) { - struct sky2_hw *hw = from_timer(hw, t, watchdog_timer); + struct sky2_hw *hw = timer_container_of(hw, t, watchdog_timer); /* Check for lost IRQ once a second */ if (sky2_read32(hw, B0_ISRC)) { diff --git a/drivers/net/ethernet/mellanox/mlx4/catas.c b/drivers/net/ethernet/mellanox/mlx4/catas.c index 33ba0a5c38ac..edcc6f662618 100644 --- a/drivers/net/ethernet/mellanox/mlx4/catas.c +++ b/drivers/net/ethernet/mellanox/mlx4/catas.c @@ -236,7 +236,7 @@ static void dump_err_buf(struct mlx4_dev *dev) static void poll_catas(struct timer_list *t) { - struct mlx4_priv *priv = from_timer(priv, t, catas_err.timer); + struct mlx4_priv *priv = timer_container_of(priv, t, catas_err.timer); struct mlx4_dev *dev = &priv->dev; u32 slave_read; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index ea078c9f5d15..3cb8d3bf9044 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -43,7 +43,6 @@ #include "en/fs_ethtool.h" #define LANES_UNKNOWN 0 -#define MAX_LANES 8 void mlx5e_ethtool_get_drvinfo(struct mlx5e_priv *priv, struct ethtool_drvinfo *drvinfo) @@ -1098,10 +1097,8 @@ static void get_link_properties(struct net_device *netdev, speed = info->speed; lanes = info->lanes; duplex = DUPLEX_FULL; - } else if (data_rate_oper) { + } else if (data_rate_oper) speed = 100 * data_rate_oper; - lanes = MAX_LANES; - } out: link_ksettings->base.duplex = duplex; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index f1d908f61134..fef418e1ed1a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -2028,9 +2028,8 @@ err_out: return err; } -static bool mlx5_flow_has_geneve_opt(struct mlx5e_tc_flow *flow) +static bool mlx5_flow_has_geneve_opt(struct mlx5_flow_spec *spec) { - struct mlx5_flow_spec *spec = &flow->attr->parse_attr->spec; void *headers_v = MLX5_ADDR_OF(fte_match_param, spec->match_value, misc_parameters_3); @@ -2069,7 +2068,7 @@ static void mlx5e_tc_del_fdb_flow(struct mlx5e_priv *priv, } complete_all(&flow->del_hw_done); - if (mlx5_flow_has_geneve_opt(flow)) + if (mlx5_flow_has_geneve_opt(&attr->parse_attr->spec)) mlx5_geneve_tlv_option_del(priv->mdev->geneve); if (flow->decap_route) @@ -2574,12 +2573,13 @@ static int parse_tunnel_attr(struct mlx5e_priv *priv, err = mlx5e_tc_tun_parse(filter_dev, priv, tmp_spec, f, match_level); if (err) { - kvfree(tmp_spec); NL_SET_ERR_MSG_MOD(extack, "Failed to parse tunnel attributes"); netdev_warn(priv->netdev, "Failed to parse tunnel attributes"); - return err; + } else { + err = mlx5e_tc_set_attr_rx_tun(flow, tmp_spec); } - err = mlx5e_tc_set_attr_rx_tun(flow, tmp_spec); + if (mlx5_flow_has_geneve_opt(tmp_spec)) + mlx5_geneve_tlv_option_del(priv->mdev->geneve); kvfree(tmp_spec); if (err) return err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 7fb8a3381f84..4917d185d0c3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1295,12 +1295,15 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw, ret = mlx5_eswitch_load_pf_vf_vport(esw, MLX5_VPORT_ECPF, enabled_events); if (ret) goto ecpf_err; - if (mlx5_core_ec_sriov_enabled(esw->dev)) { - ret = mlx5_eswitch_load_ec_vf_vports(esw, esw->esw_funcs.num_ec_vfs, - enabled_events); - if (ret) - goto ec_vf_err; - } + } + + /* Enable ECVF vports */ + if (mlx5_core_ec_sriov_enabled(esw->dev)) { + ret = mlx5_eswitch_load_ec_vf_vports(esw, + esw->esw_funcs.num_ec_vfs, + enabled_events); + if (ret) + goto ec_vf_err; } /* Enable VF vports */ @@ -1331,9 +1334,11 @@ void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw) { mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs); + if (mlx5_core_ec_sriov_enabled(esw->dev)) + mlx5_eswitch_unload_ec_vf_vports(esw, + esw->esw_funcs.num_ec_vfs); + if (mlx5_ecpf_vport_exists(esw->dev)) { - if (mlx5_core_ec_sriov_enabled(esw->dev)) - mlx5_eswitch_unload_ec_vf_vports(esw, esw->esw_funcs.num_vfs); mlx5_eswitch_unload_pf_vf_vport(esw, MLX5_VPORT_ECPF); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index 23a7e8e7adfa..a8046200d376 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -2228,6 +2228,7 @@ try_add_to_existing_fg(struct mlx5_flow_table *ft, struct mlx5_flow_handle *rule; struct match_list *iter; bool take_write = false; + bool try_again = false; struct fs_fte *fte; u64 version = 0; int err; @@ -2292,6 +2293,7 @@ skip_search: nested_down_write_ref_node(&g->node, FS_LOCK_PARENT); if (!g->node.active) { + try_again = true; up_write_ref_node(&g->node, false); continue; } @@ -2313,7 +2315,8 @@ skip_search: tree_put_node(&fte->node, false); return rule; } - rule = ERR_PTR(-ENOENT); + err = try_again ? -EAGAIN : -ENOENT; + rule = ERR_PTR(err); out: kmem_cache_free(steering->ftes_cache, fte); return rule; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 5442a02c4097..69933addd921 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -278,7 +278,8 @@ static void mlx5_sync_reset_reload_work(struct work_struct *work) #define MLX5_RESET_POLL_INTERVAL (HZ / 10) static void poll_sync_reset(struct timer_list *t) { - struct mlx5_fw_reset *fw_reset = from_timer(fw_reset, t, timer); + struct mlx5_fw_reset *fw_reset = timer_container_of(fw_reset, t, + timer); struct mlx5_core_dev *dev = fw_reset->dev; u32 fatal_error; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index 624452ddebc0..cf7a1edd0530 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -779,7 +779,8 @@ static void mlx5_health_log_ts_update(struct work_struct *work) static void poll_health(struct timer_list *t) { - struct mlx5_core_dev *dev = from_timer(dev, t, priv.health.timer); + struct mlx5_core_dev *dev = timer_container_of(dev, t, + priv.health.timer); struct mlx5_core_health *health = &dev->priv.health; struct health_buffer __iomem *h = health->health; u32 fatal_error; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c index 972e8e9df585..9bc9bd83c232 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c @@ -291,7 +291,7 @@ static void free_4k(struct mlx5_core_dev *dev, u64 addr, u32 function) static int alloc_system_page(struct mlx5_core_dev *dev, u32 function) { struct device *device = mlx5_core_dma_dev(dev); - int nid = dev_to_node(device); + int nid = dev->priv.numa_node; struct page *page; u64 zero_addr = 1; u64 addr; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/action.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/action.c index fb62f3bc4bd4..447ea3f8722c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/action.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/action.c @@ -1370,8 +1370,8 @@ mlx5hws_action_create_dest_array(struct mlx5hws_context *ctx, struct mlx5hws_cmd_set_fte_attr fte_attr = {0}; struct mlx5hws_cmd_forward_tbl *fw_island; struct mlx5hws_action *action; - u32 i /*, packet_reformat_id*/; - int ret; + int ret, last_dest_idx = -1; + u32 i; if (num_dest <= 1) { mlx5hws_err(ctx, "Action must have multiple dests\n"); @@ -1401,11 +1401,8 @@ mlx5hws_action_create_dest_array(struct mlx5hws_context *ctx, dest_list[i].destination_id = dests[i].dest->dest_obj.obj_id; fte_attr.action_flags |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST; fte_attr.ignore_flow_level = ignore_flow_level; - /* ToDo: In SW steering we have a handling of 'go to WIRE' - * destination here by upper layer setting 'is_wire_ft' flag - * if the destination is wire. - * This is because uplink should be last dest in the list. - */ + if (dests[i].is_wire_ft) + last_dest_idx = i; break; case MLX5HWS_ACTION_TYP_VPORT: dest_list[i].destination_type = MLX5_FLOW_DESTINATION_TYPE_VPORT; @@ -1429,6 +1426,9 @@ mlx5hws_action_create_dest_array(struct mlx5hws_context *ctx, } } + if (last_dest_idx != -1) + swap(dest_list[last_dest_idx], dest_list[num_dest - 1]); + fte_attr.dests_num = num_dest; fte_attr.dests = dest_list; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c index 70768953a4f6..ca7501c57468 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c @@ -1070,7 +1070,7 @@ hws_bwc_rule_complex_hash_node_get(struct mlx5hws_bwc_rule *bwc_rule, struct mlx5hws_bwc_matcher *bwc_matcher = bwc_rule->bwc_matcher; struct mlx5hws_bwc_complex_rule_hash_node *node, *old_node; struct rhashtable *refcount_hash; - int i; + int ret, i; bwc_rule->complex_hash_node = NULL; @@ -1078,7 +1078,11 @@ hws_bwc_rule_complex_hash_node_get(struct mlx5hws_bwc_rule *bwc_rule, if (unlikely(!node)) return -ENOMEM; - node->tag = ida_alloc(&bwc_matcher->complex->metadata_ida, GFP_KERNEL); + ret = ida_alloc(&bwc_matcher->complex->metadata_ida, GFP_KERNEL); + if (ret < 0) + goto err_free_node; + node->tag = ret; + refcount_set(&node->refcount, 1); /* Clear match buffer - turn off all the unrelated fields @@ -1094,6 +1098,11 @@ hws_bwc_rule_complex_hash_node_get(struct mlx5hws_bwc_rule *bwc_rule, old_node = rhashtable_lookup_get_insert_fast(refcount_hash, &node->hash_node, hws_refcount_hash); + if (IS_ERR(old_node)) { + ret = PTR_ERR(old_node); + goto err_free_ida; + } + if (old_node) { /* Rule with the same tag already exists - update refcount */ refcount_inc(&old_node->refcount); @@ -1112,6 +1121,12 @@ hws_bwc_rule_complex_hash_node_get(struct mlx5hws_bwc_rule *bwc_rule, bwc_rule->complex_hash_node = node; return 0; + +err_free_ida: + ida_free(&bwc_matcher->complex->metadata_ida, node->tag); +err_free_node: + kfree(node); + return ret; } static void diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/definer.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/definer.c index 5cc0dc002ac1..d45e1145d197 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/definer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/definer.c @@ -785,6 +785,9 @@ hws_definer_conv_outer(struct mlx5hws_definer_conv_data *cd, HWS_SET_HDR(fc, match_param, IP_PROTOCOL_O, outer_headers.ip_protocol, eth_l3_outer.protocol_next_header); + HWS_SET_HDR(fc, match_param, IP_VERSION_O, + outer_headers.ip_version, + eth_l3_outer.ip_version); HWS_SET_HDR(fc, match_param, IP_TTL_O, outer_headers.ttl_hoplimit, eth_l3_outer.time_to_live_hop_limit); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c index 9d1c0e4b224a..bf4643d0ce17 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c @@ -966,6 +966,9 @@ static int mlx5_fs_fte_get_hws_actions(struct mlx5_flow_root_namespace *ns, switch (attr->type) { case MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE: dest_action = mlx5_fs_get_dest_action_ft(fs_ctx, dst); + if (dst->dest_attr.ft->flags & + MLX5_FLOW_TABLE_UPLINK_VPORT) + dest_actions[num_dest_actions].is_wire_ft = true; break; case MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE_NUM: dest_action = mlx5_fs_get_dest_action_table_num(fs_ctx, @@ -1357,6 +1360,7 @@ mlx5_cmd_hws_packet_reformat_alloc(struct mlx5_flow_root_namespace *ns, pkt_reformat->fs_hws_action.pr_data = pr_data; } + mutex_init(&pkt_reformat->fs_hws_action.lock); pkt_reformat->owner = MLX5_FLOW_RESOURCE_OWNER_HWS; pkt_reformat->fs_hws_action.hws_action = hws_action; return 0; @@ -1503,7 +1507,6 @@ static int mlx5_cmd_hws_modify_header_alloc(struct mlx5_flow_root_namespace *ns, err = -ENOMEM; goto release_mh; } - mutex_init(&modify_hdr->fs_hws_action.lock); modify_hdr->fs_hws_action.mh_data = mh_data; modify_hdr->fs_hws_action.fs_pool = pool; modify_hdr->owner = MLX5_FLOW_RESOURCE_OWNER_SW; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h index 9bbadc4d8a0b..d8ac6c196211 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h @@ -213,6 +213,7 @@ struct mlx5hws_action_dest_attr { struct mlx5hws_action *dest; /* Optional reformat action */ struct mlx5hws_action *reformat; + bool is_wire_ft; }; /** diff --git a/drivers/net/ethernet/micrel/ksz884x.c b/drivers/net/ethernet/micrel/ksz884x.c index 1302aa8e0853..cdde19b8edc4 100644 --- a/drivers/net/ethernet/micrel/ksz884x.c +++ b/drivers/net/ethernet/micrel/ksz884x.c @@ -6304,7 +6304,8 @@ static void mib_read_work(struct work_struct *work) static void mib_monitor(struct timer_list *t) { - struct dev_info *hw_priv = from_timer(hw_priv, t, mib_timer_info.timer); + struct dev_info *hw_priv = timer_container_of(hw_priv, t, + mib_timer_info.timer); mib_read_work(&hw_priv->mib_read); @@ -6331,7 +6332,8 @@ static void mib_monitor(struct timer_list *t) */ static void dev_monitor(struct timer_list *t) { - struct dev_priv *priv = from_timer(priv, t, monitor_timer_info.timer); + struct dev_priv *priv = timer_container_of(priv, t, + monitor_timer_info.timer); struct net_device *dev = priv->mii_if.dev; struct dev_info *hw_priv = priv->adapter; struct ksz_hw *hw = &hw_priv->hw; diff --git a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c index 7c501a758325..e611ff7fa3fa 100644 --- a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c +++ b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c @@ -3478,7 +3478,7 @@ static void myri10ge_watchdog_timer(struct timer_list *t) u32 rx_pause_cnt; u16 cmd; - mgp = from_timer(mgp, t, watchdog_timer); + mgp = timer_container_of(mgp, t, watchdog_timer); rx_pause_cnt = ntohl(mgp->ss[0].fw_stats->dropped_pause); busy_slice_cnt = 0; diff --git a/drivers/net/ethernet/natsemi/natsemi.c b/drivers/net/ethernet/natsemi/natsemi.c index dd279788cf9e..b253734dbc80 100644 --- a/drivers/net/ethernet/natsemi/natsemi.c +++ b/drivers/net/ethernet/natsemi/natsemi.c @@ -1786,7 +1786,7 @@ static void init_registers(struct net_device *dev) */ static void netdev_timer(struct timer_list *t) { - struct netdev_private *np = from_timer(np, t, timer); + struct netdev_private *np = timer_container_of(np, t, timer); struct net_device *dev = np->dev; void __iomem * ioaddr = ns_ioaddr(dev); int next_tick = NATSEMI_TIMER_FREQ; diff --git a/drivers/net/ethernet/natsemi/ns83820.c b/drivers/net/ethernet/natsemi/ns83820.c index bf0347715a05..56d5464222d9 100644 --- a/drivers/net/ethernet/natsemi/ns83820.c +++ b/drivers/net/ethernet/natsemi/ns83820.c @@ -1587,7 +1587,7 @@ static void ns83820_tx_timeout(struct net_device *ndev, unsigned int txqueue) static void ns83820_tx_watch(struct timer_list *t) { - struct ns83820 *dev = from_timer(dev, t, tx_watchdog); + struct ns83820 *dev = timer_container_of(dev, t, tx_watchdog); struct net_device *ndev = dev->ndev; #if defined(DEBUG) diff --git a/drivers/net/ethernet/neterion/s2io.c b/drivers/net/ethernet/neterion/s2io.c index 3e55e8dc664c..27443e346f9f 100644 --- a/drivers/net/ethernet/neterion/s2io.c +++ b/drivers/net/ethernet/neterion/s2io.c @@ -4195,7 +4195,7 @@ pci_map_failed: static void s2io_alarm_handle(struct timer_list *t) { - struct s2io_nic *sp = from_timer(sp, t, alarm_timer); + struct s2io_nic *sp = timer_container_of(sp, t, alarm_timer); struct net_device *dev = sp->dev; s2io_handle_errors(dev); diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 28997ddab966..932f59d70f41 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -159,7 +159,7 @@ static int nfp_net_reconfig_wait(struct nfp_net *nn, unsigned long deadline) static void nfp_net_reconfig_timer(struct timer_list *t) { - struct nfp_net *nn = from_timer(nn, t, reconfig_timer); + struct nfp_net *nn = timer_container_of(nn, t, reconfig_timer); spin_lock_bh(&nn->reconfig_lock); diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c index 29cb74ccb25a..19aa1f1538aa 100644 --- a/drivers/net/ethernet/nvidia/forcedeth.c +++ b/drivers/net/ethernet/nvidia/forcedeth.c @@ -1891,7 +1891,7 @@ packet_dropped: /* If rx bufs are exhausted called after 50ms to attempt to refresh */ static void nv_do_rx_refill(struct timer_list *t) { - struct fe_priv *np = from_timer(np, t, oom_kick); + struct fe_priv *np = timer_container_of(np, t, oom_kick); /* Just reschedule NAPI rx processing */ napi_schedule(&np->napi); @@ -4140,7 +4140,7 @@ static void nv_free_irq(struct net_device *dev) static void nv_do_nic_poll(struct timer_list *t) { - struct fe_priv *np = from_timer(np, t, nic_poll); + struct fe_priv *np = timer_container_of(np, t, nic_poll); struct net_device *dev = np->dev; u8 __iomem *base = get_hwbase(dev); u32 mask = 0; @@ -4259,7 +4259,7 @@ static void nv_do_stats_poll(struct timer_list *t) __acquires(&netdev_priv(dev)->hwstats_lock) __releases(&netdev_priv(dev)->hwstats_lock) { - struct fe_priv *np = from_timer(np, t, stats_poll); + struct fe_priv *np = timer_container_of(np, t, stats_poll); struct net_device *dev = np->dev; /* If lock is currently taken, the stats are being refreshed diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c index 1651df8a7c21..e5a6f59af0b6 100644 --- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c +++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c @@ -1014,8 +1014,8 @@ static void pch_gbe_set_mode(struct pch_gbe_adapter *adapter, u16 speed, */ static void pch_gbe_watchdog(struct timer_list *t) { - struct pch_gbe_adapter *adapter = from_timer(adapter, t, - watchdog_timer); + struct pch_gbe_adapter *adapter = timer_container_of(adapter, t, + watchdog_timer); struct net_device *netdev = adapter->netdev; struct pch_gbe_hw *hw = &adapter->hw; diff --git a/drivers/net/ethernet/packetengines/hamachi.c b/drivers/net/ethernet/packetengines/hamachi.c index 26bc8b3b1ec8..b0de7e9f12a5 100644 --- a/drivers/net/ethernet/packetengines/hamachi.c +++ b/drivers/net/ethernet/packetengines/hamachi.c @@ -1025,7 +1025,7 @@ static inline int hamachi_tx(struct net_device *dev) static void hamachi_timer(struct timer_list *t) { - struct hamachi_private *hmp = from_timer(hmp, t, timer); + struct hamachi_private *hmp = timer_container_of(hmp, t, timer); struct net_device *dev = hmp->mii_if.dev; void __iomem *ioaddr = hmp->base; int next_tick = 10*HZ; diff --git a/drivers/net/ethernet/packetengines/yellowfin.c b/drivers/net/ethernet/packetengines/yellowfin.c index 21b760e65d73..1e25ac13a7d8 100644 --- a/drivers/net/ethernet/packetengines/yellowfin.c +++ b/drivers/net/ethernet/packetengines/yellowfin.c @@ -652,7 +652,7 @@ err_free_irq: static void yellowfin_timer(struct timer_list *t) { - struct yellowfin_private *yp = from_timer(yp, t, timer); + struct yellowfin_private *yp = timer_container_of(yp, t, timer); struct net_device *dev = pci_get_drvdata(yp->pci_dev); void __iomem *ioaddr = yp->base; int next_tick = 60*HZ; diff --git a/drivers/net/ethernet/pasemi/pasemi_mac.c b/drivers/net/ethernet/pasemi/pasemi_mac.c index 801380729046..fe58024b5901 100644 --- a/drivers/net/ethernet/pasemi/pasemi_mac.c +++ b/drivers/net/ethernet/pasemi/pasemi_mac.c @@ -934,7 +934,8 @@ static irqreturn_t pasemi_mac_rx_intr(int irq, void *data) static void pasemi_mac_tx_timer(struct timer_list *t) { - struct pasemi_mac_txring *txring = from_timer(txring, t, clean_timer); + struct pasemi_mac_txring *txring = timer_container_of(txring, t, + clean_timer); struct pasemi_mac *mac = txring->mac; pasemi_mac_clean_tx(txring); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.c b/drivers/net/ethernet/pensando/ionic/ionic_dev.c index 57edcde9e6f8..18b9c8a810ae 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.c @@ -13,7 +13,7 @@ static void ionic_watchdog_cb(struct timer_list *t) { - struct ionic *ionic = from_timer(ionic, t, watchdog_timer); + struct ionic *ionic = timer_container_of(ionic, t, watchdog_timer); struct ionic_lif *lif = ionic->lif; struct ionic_deferred_work *work; int hb; diff --git a/drivers/net/ethernet/qlogic/qla3xxx.c b/drivers/net/ethernet/qlogic/qla3xxx.c index bf5bf8c95c85..aee4e63b4b82 100644 --- a/drivers/net/ethernet/qlogic/qla3xxx.c +++ b/drivers/net/ethernet/qlogic/qla3xxx.c @@ -3735,7 +3735,7 @@ static void ql_get_board_info(struct ql3_adapter *qdev) static void ql3xxx_timer(struct timer_list *t) { - struct ql3_adapter *qdev = from_timer(qdev, t, adapter_timer); + struct ql3_adapter *qdev = timer_container_of(qdev, t, adapter_timer); queue_delayed_work(qdev->workqueue, &qdev->link_state_work, 0); } diff --git a/drivers/net/ethernet/realtek/atp.c b/drivers/net/ethernet/realtek/atp.c index c73a57e4a144..0d65434982a2 100644 --- a/drivers/net/ethernet/realtek/atp.c +++ b/drivers/net/ethernet/realtek/atp.c @@ -717,7 +717,7 @@ static irqreturn_t atp_interrupt(int irq, void *dev_instance) problem where the adapter forgets its ethernet address. */ static void atp_timed_checker(struct timer_list *t) { - struct net_local *lp = from_timer(lp, t, timer); + struct net_local *lp = timer_container_of(lp, t, timer); struct net_device *dev = lp->dev; long ioaddr = dev->base_addr; int tickssofar = jiffies - lp->last_rx_time; diff --git a/drivers/net/ethernet/rocker/rocker_ofdpa.c b/drivers/net/ethernet/rocker/rocker_ofdpa.c index 19985d13e243..61e50517c05b 100644 --- a/drivers/net/ethernet/rocker/rocker_ofdpa.c +++ b/drivers/net/ethernet/rocker/rocker_ofdpa.c @@ -1982,7 +1982,7 @@ err_out: static void ofdpa_fdb_cleanup(struct timer_list *t) { - struct ofdpa *ofdpa = from_timer(ofdpa, t, fdb_cleanup_timer); + struct ofdpa *ofdpa = timer_container_of(ofdpa, t, fdb_cleanup_timer); struct ofdpa_port *ofdpa_port; struct ofdpa_fdb_tbl_entry *entry; struct hlist_node *tmp; diff --git a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c index 36b63bf343a9..75bad561b352 100644 --- a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c +++ b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c @@ -104,7 +104,8 @@ void sxgbe_disable_eee_mode(struct sxgbe_priv_data * const priv) */ static void sxgbe_eee_ctrl_timer(struct timer_list *t) { - struct sxgbe_priv_data *priv = from_timer(priv, t, eee_ctrl_timer); + struct sxgbe_priv_data *priv = timer_container_of(priv, t, + eee_ctrl_timer); sxgbe_enable_eee_mode(priv); mod_timer(&priv->eee_ctrl_timer, SXGBE_LPI_TIMER(eee_timer)); @@ -1012,7 +1013,7 @@ static void sxgbe_disable_mtl_engine(struct sxgbe_priv_data *priv) */ static void sxgbe_tx_timer(struct timer_list *t) { - struct sxgbe_tx_queue *p = from_timer(p, t, txtimer); + struct sxgbe_tx_queue *p = timer_container_of(p, t, txtimer); sxgbe_tx_queue_clean(p); } diff --git a/drivers/net/ethernet/seeq/ether3.c b/drivers/net/ethernet/seeq/ether3.c index 1d65113fab76..20dad39b5ab9 100644 --- a/drivers/net/ethernet/seeq/ether3.c +++ b/drivers/net/ethernet/seeq/ether3.c @@ -170,7 +170,7 @@ ether3_setbuffer(struct net_device *dev, buffer_rw_t read, int start) */ static void ether3_ledoff(struct timer_list *t) { - struct dev_priv *private = from_timer(private, t, timer); + struct dev_priv *private = timer_container_of(private, t, timer); struct net_device *dev = private->dev; ether3_outw(priv(dev)->regs.config2 |= CFG2_CTRLO, REG_CONFIG2); diff --git a/drivers/net/ethernet/sfc/falcon/falcon.c b/drivers/net/ethernet/sfc/falcon/falcon.c index b865275beb66..c44df8e4dd30 100644 --- a/drivers/net/ethernet/sfc/falcon/falcon.c +++ b/drivers/net/ethernet/sfc/falcon/falcon.c @@ -1453,8 +1453,8 @@ static void falcon_stats_complete(struct ef4_nic *efx) static void falcon_stats_timer_func(struct timer_list *t) { - struct falcon_nic_data *nic_data = from_timer(nic_data, t, - stats_timer); + struct falcon_nic_data *nic_data = timer_container_of(nic_data, t, + stats_timer); struct ef4_nic *efx = nic_data->efx; spin_lock(&efx->stats_lock); diff --git a/drivers/net/ethernet/sfc/falcon/rx.c b/drivers/net/ethernet/sfc/falcon/rx.c index 38ad7ac07726..f69fcf6caca8 100644 --- a/drivers/net/ethernet/sfc/falcon/rx.c +++ b/drivers/net/ethernet/sfc/falcon/rx.c @@ -382,7 +382,8 @@ void ef4_fast_push_rx_descriptors(struct ef4_rx_queue *rx_queue, bool atomic) void ef4_rx_slow_fill(struct timer_list *t) { - struct ef4_rx_queue *rx_queue = from_timer(rx_queue, t, slow_fill); + struct ef4_rx_queue *rx_queue = timer_container_of(rx_queue, t, + slow_fill); /* Post an event to cause NAPI to run and refill the queue */ ef4_nic_generate_fill_event(rx_queue); diff --git a/drivers/net/ethernet/sfc/mcdi.c b/drivers/net/ethernet/sfc/mcdi.c index dcef0588be96..5e9b8def5e42 100644 --- a/drivers/net/ethernet/sfc/mcdi.c +++ b/drivers/net/ethernet/sfc/mcdi.c @@ -605,7 +605,7 @@ static void efx_mcdi_ev_cpl(struct efx_nic *efx, unsigned int seqno, static void efx_mcdi_timeout_async(struct timer_list *t) { - struct efx_mcdi_iface *mcdi = from_timer(mcdi, t, async_timer); + struct efx_mcdi_iface *mcdi = timer_container_of(mcdi, t, async_timer); efx_mcdi_complete_async(mcdi, true); } diff --git a/drivers/net/ethernet/sfc/rx_common.c b/drivers/net/ethernet/sfc/rx_common.c index 8eb272ba674b..f4f75299dfa9 100644 --- a/drivers/net/ethernet/sfc/rx_common.c +++ b/drivers/net/ethernet/sfc/rx_common.c @@ -352,7 +352,8 @@ void efx_free_rx_buffers(struct efx_rx_queue *rx_queue, void efx_rx_slow_fill(struct timer_list *t) { - struct efx_rx_queue *rx_queue = from_timer(rx_queue, t, slow_fill); + struct efx_rx_queue *rx_queue = timer_container_of(rx_queue, t, + slow_fill); /* Post an event to cause NAPI to run and refill the queue */ efx_nic_generate_fill_event(rx_queue); diff --git a/drivers/net/ethernet/sfc/siena/mcdi.c b/drivers/net/ethernet/sfc/siena/mcdi.c index 99ab5f294691..c8f0fb43e285 100644 --- a/drivers/net/ethernet/sfc/siena/mcdi.c +++ b/drivers/net/ethernet/sfc/siena/mcdi.c @@ -609,7 +609,7 @@ static void efx_mcdi_ev_cpl(struct efx_nic *efx, unsigned int seqno, static void efx_mcdi_timeout_async(struct timer_list *t) { - struct efx_mcdi_iface *mcdi = from_timer(mcdi, t, async_timer); + struct efx_mcdi_iface *mcdi = timer_container_of(mcdi, t, async_timer); efx_mcdi_complete_async(mcdi, true); } diff --git a/drivers/net/ethernet/sfc/siena/rx_common.c b/drivers/net/ethernet/sfc/siena/rx_common.c index ab493e529d5c..98d27174015d 100644 --- a/drivers/net/ethernet/sfc/siena/rx_common.c +++ b/drivers/net/ethernet/sfc/siena/rx_common.c @@ -349,7 +349,8 @@ void efx_siena_free_rx_buffers(struct efx_rx_queue *rx_queue, void efx_siena_rx_slow_fill(struct timer_list *t) { - struct efx_rx_queue *rx_queue = from_timer(rx_queue, t, slow_fill); + struct efx_rx_queue *rx_queue = timer_container_of(rx_queue, t, + slow_fill); /* Post an event to cause NAPI to run and refill the queue */ efx_nic_generate_fill_event(rx_queue); diff --git a/drivers/net/ethernet/sgi/ioc3-eth.c b/drivers/net/ethernet/sgi/ioc3-eth.c index 7196e1c607f3..39731069d99e 100644 --- a/drivers/net/ethernet/sgi/ioc3-eth.c +++ b/drivers/net/ethernet/sgi/ioc3-eth.c @@ -570,7 +570,7 @@ static inline void ioc3_setup_duplex(struct ioc3_private *ip) static void ioc3_timer(struct timer_list *t) { - struct ioc3_private *ip = from_timer(ip, t, ioc3_timer); + struct ioc3_private *ip = timer_container_of(ip, t, ioc3_timer); /* Print the link status if it has changed */ mii_check_media(&ip->mii, 1, 0); diff --git a/drivers/net/ethernet/sis/sis190.c b/drivers/net/ethernet/sis/sis190.c index d10b14787607..15e46e6ac262 100644 --- a/drivers/net/ethernet/sis/sis190.c +++ b/drivers/net/ethernet/sis/sis190.c @@ -1023,7 +1023,7 @@ out_unlock: static void sis190_phy_timer(struct timer_list *t) { - struct sis190_private *tp = from_timer(tp, t, timer); + struct sis190_private *tp = timer_container_of(tp, t, timer); struct net_device *dev = tp->dev; if (likely(netif_running(dev))) diff --git a/drivers/net/ethernet/sis/sis900.c b/drivers/net/ethernet/sis/sis900.c index df869f82cae8..b461918dc5f4 100644 --- a/drivers/net/ethernet/sis/sis900.c +++ b/drivers/net/ethernet/sis/sis900.c @@ -1314,7 +1314,8 @@ static void sis630_set_eq(struct net_device *net_dev, u8 revision) static void sis900_timer(struct timer_list *t) { - struct sis900_private *sis_priv = from_timer(sis_priv, t, timer); + struct sis900_private *sis_priv = timer_container_of(sis_priv, t, + timer); struct net_device *net_dev = sis_priv->mii_info.dev; struct mii_phy *mii_phy = sis_priv->mii; static const int next_tick = 5*HZ; diff --git a/drivers/net/ethernet/smsc/epic100.c b/drivers/net/ethernet/smsc/epic100.c index ca0ab3a35b73..45f703fe0e5a 100644 --- a/drivers/net/ethernet/smsc/epic100.c +++ b/drivers/net/ethernet/smsc/epic100.c @@ -850,7 +850,7 @@ static void check_media(struct net_device *dev) static void epic_timer(struct timer_list *t) { - struct epic_private *ep = from_timer(ep, t, timer); + struct epic_private *ep = timer_container_of(ep, t, timer); struct net_device *dev = ep->mii.dev; void __iomem *ioaddr = ep->ioaddr; int next_tick = 5*HZ; diff --git a/drivers/net/ethernet/smsc/smc91c92_cs.c b/drivers/net/ethernet/smsc/smc91c92_cs.c index 6fa957fb523b..cc0c75694351 100644 --- a/drivers/net/ethernet/smsc/smc91c92_cs.c +++ b/drivers/net/ethernet/smsc/smc91c92_cs.c @@ -1713,7 +1713,7 @@ static void smc_reset(struct net_device *dev) static void media_check(struct timer_list *t) { - struct smc_private *smc = from_timer(smc, t, media); + struct smc_private *smc = timer_container_of(smc, t, media); struct net_device *dev = smc->mii_if.dev; unsigned int ioaddr = dev->base_addr; u_short i, media, saved_bank; diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 1369fa70bc58..b948df1bff9a 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -481,7 +481,7 @@ static void stmmac_stop_sw_lpi(struct stmmac_priv *priv) */ static void stmmac_eee_ctrl_timer(struct timer_list *t) { - struct stmmac_priv *priv = from_timer(priv, t, eee_ctrl_timer); + struct stmmac_priv *priv = timer_container_of(priv, t, eee_ctrl_timer); stmmac_try_to_start_sw_lpi(priv); } diff --git a/drivers/net/ethernet/sun/cassini.c b/drivers/net/ethernet/sun/cassini.c index b777e5a099eb..acfb523214b9 100644 --- a/drivers/net/ethernet/sun/cassini.c +++ b/drivers/net/ethernet/sun/cassini.c @@ -4021,7 +4021,7 @@ done: static void cas_link_timer(struct timer_list *t) { - struct cas *cp = from_timer(cp, t, link_timer); + struct cas *cp = timer_container_of(cp, t, link_timer); int mask, pending = 0, reset = 0; unsigned long flags; diff --git a/drivers/net/ethernet/sun/niu.c b/drivers/net/ethernet/sun/niu.c index 379b6e90121d..ddca8fc7883e 100644 --- a/drivers/net/ethernet/sun/niu.c +++ b/drivers/net/ethernet/sun/niu.c @@ -2225,7 +2225,7 @@ static int niu_link_status(struct niu *np, int *link_up_p) static void niu_timer(struct timer_list *t) { - struct niu *np = from_timer(np, t, timer); + struct niu *np = timer_container_of(np, t, timer); unsigned long off; int err, link_up; diff --git a/drivers/net/ethernet/sun/sunbmac.c b/drivers/net/ethernet/sun/sunbmac.c index d2c82102133c..edb2fd3a6551 100644 --- a/drivers/net/ethernet/sun/sunbmac.c +++ b/drivers/net/ethernet/sun/sunbmac.c @@ -526,7 +526,7 @@ static int try_next_permutation(struct bigmac *bp, void __iomem *tregs) static void bigmac_timer(struct timer_list *t) { - struct bigmac *bp = from_timer(bp, t, bigmac_timer); + struct bigmac *bp = timer_container_of(bp, t, bigmac_timer); void __iomem *tregs = bp->tregs; int restart_timer = 0; diff --git a/drivers/net/ethernet/sun/sungem.c b/drivers/net/ethernet/sun/sungem.c index 06579d7b5220..8e69d917d827 100644 --- a/drivers/net/ethernet/sun/sungem.c +++ b/drivers/net/ethernet/sun/sungem.c @@ -1481,7 +1481,7 @@ static int gem_mdio_link_not_up(struct gem *gp) static void gem_link_timer(struct timer_list *t) { - struct gem *gp = from_timer(gp, t, link_timer); + struct gem *gp = timer_container_of(gp, t, link_timer); struct net_device *dev = gp->dev; int restart_aneg = 0; diff --git a/drivers/net/ethernet/sun/sunhme.c b/drivers/net/ethernet/sun/sunhme.c index 9a7586623318..4bc0e114d5ee 100644 --- a/drivers/net/ethernet/sun/sunhme.c +++ b/drivers/net/ethernet/sun/sunhme.c @@ -721,7 +721,7 @@ force_link: static void happy_meal_timer(struct timer_list *t) { - struct happy_meal *hp = from_timer(hp, t, happy_timer); + struct happy_meal *hp = timer_container_of(hp, t, happy_timer); void __iomem *tregs = hp->tcvregs; int restart_timer = 0; diff --git a/drivers/net/ethernet/sun/sunvnet_common.c b/drivers/net/ethernet/sun/sunvnet_common.c index ddc6d46a7a86..0212853c9430 100644 --- a/drivers/net/ethernet/sun/sunvnet_common.c +++ b/drivers/net/ethernet/sun/sunvnet_common.c @@ -1044,7 +1044,7 @@ static inline void vnet_free_skbs(struct sk_buff *skb) void sunvnet_clean_timer_expire_common(struct timer_list *t) { - struct vnet_port *port = from_timer(port, t, clean_timer); + struct vnet_port *port = timer_container_of(port, t, clean_timer); struct sk_buff *freeskbs; unsigned pending; diff --git a/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c b/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c index 24e4b246f25f..37101dc50d04 100644 --- a/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c +++ b/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c @@ -360,7 +360,8 @@ static irqreturn_t xlgmac_dma_isr(int irq, void *data) static void xlgmac_tx_timer(struct timer_list *t) { - struct xlgmac_channel *channel = from_timer(channel, t, tx_timer); + struct xlgmac_channel *channel = timer_container_of(channel, t, + tx_timer); struct xlgmac_pdata *pdata = channel->pdata; struct napi_struct *napi; diff --git a/drivers/net/ethernet/ti/cpsw_ale.c b/drivers/net/ethernet/ti/cpsw_ale.c index 7f77694ecfba..fbe35af615a6 100644 --- a/drivers/net/ethernet/ti/cpsw_ale.c +++ b/drivers/net/ethernet/ti/cpsw_ale.c @@ -1231,7 +1231,7 @@ int cpsw_ale_rx_ratelimit_bc(struct cpsw_ale *ale, int port, unsigned int rateli static void cpsw_ale_timer(struct timer_list *t) { - struct cpsw_ale *ale = from_timer(ale, t, timer); + struct cpsw_ale *ale = timer_container_of(ale, t, timer); cpsw_ale_control_set(ale, 0, ALE_AGEOUT, 1); diff --git a/drivers/net/ethernet/ti/netcp_ethss.c b/drivers/net/ethernet/ti/netcp_ethss.c index fd2b74508980..55a1a96cd834 100644 --- a/drivers/net/ethernet/ti/netcp_ethss.c +++ b/drivers/net/ethernet/ti/netcp_ethss.c @@ -2833,7 +2833,7 @@ static int gbe_ioctl(void *intf_priv, struct ifreq *req, int cmd) static void netcp_ethss_timer(struct timer_list *t) { - struct gbe_priv *gbe_dev = from_timer(gbe_dev, t, timer); + struct gbe_priv *gbe_dev = timer_container_of(gbe_dev, t, timer); struct gbe_intf *gbe_intf; struct gbe_slave *slave; diff --git a/drivers/net/ethernet/ti/tlan.c b/drivers/net/ethernet/ti/tlan.c index d9240fb91747..a55b0f951181 100644 --- a/drivers/net/ethernet/ti/tlan.c +++ b/drivers/net/ethernet/ti/tlan.c @@ -1815,7 +1815,7 @@ ThunderLAN driver timer function static void tlan_timer(struct timer_list *t) { - struct tlan_priv *priv = from_timer(priv, t, timer); + struct tlan_priv *priv = timer_container_of(priv, t, timer); struct net_device *dev = priv->dev; u32 elapsed; unsigned long flags = 0; @@ -2746,7 +2746,7 @@ static void tlan_phy_finish_auto_neg(struct net_device *dev) static void tlan_phy_monitor(struct timer_list *t) { - struct tlan_priv *priv = from_timer(priv, t, media_timer); + struct tlan_priv *priv = timer_container_of(priv, t, media_timer); struct net_device *dev = priv->dev; u16 phy; u16 phy_status; diff --git a/drivers/net/ethernet/tundra/tsi108_eth.c b/drivers/net/ethernet/tundra/tsi108_eth.c index 7ec0e3c13d54..7e0b3d694ac0 100644 --- a/drivers/net/ethernet/tundra/tsi108_eth.c +++ b/drivers/net/ethernet/tundra/tsi108_eth.c @@ -1652,7 +1652,7 @@ regs_fail: static void tsi108_timed_checker(struct timer_list *t) { - struct tsi108_prv_data *data = from_timer(data, t, timer); + struct tsi108_prv_data *data = timer_container_of(data, t, timer); struct net_device *dev = data->dev; tsi108_check_phy(dev); diff --git a/drivers/net/ethernet/wangxun/libwx/wx_lib.c b/drivers/net/ethernet/wangxun/libwx/wx_lib.c index 5c747509d56b..7f2e6cddfeb1 100644 --- a/drivers/net/ethernet/wangxun/libwx/wx_lib.c +++ b/drivers/net/ethernet/wangxun/libwx/wx_lib.c @@ -3116,7 +3116,7 @@ EXPORT_SYMBOL(wx_service_event_complete); void wx_service_timer(struct timer_list *t) { - struct wx *wx = from_timer(wx, t, service_timer); + struct wx *wx = timer_container_of(wx, t, service_timer); unsigned long next_event_offset = HZ * 2; /* Reset the timer */ diff --git a/drivers/net/fddi/defza.c b/drivers/net/fddi/defza.c index 54b7f24f3810..2098c3068d34 100644 --- a/drivers/net/fddi/defza.c +++ b/drivers/net/fddi/defza.c @@ -1044,7 +1044,7 @@ static irqreturn_t fza_interrupt(int irq, void *dev_id) static void fza_reset_timer(struct timer_list *t) { - struct fza_private *fp = from_timer(fp, t, reset_timer); + struct fza_private *fp = timer_container_of(fp, t, reset_timer); if (!fp->timer_state) { pr_err("%s: RESET timed out!\n", fp->name); diff --git a/drivers/net/hamradio/6pack.c b/drivers/net/hamradio/6pack.c index b33d84ed5bbf..c5e5423e1863 100644 --- a/drivers/net/hamradio/6pack.c +++ b/drivers/net/hamradio/6pack.c @@ -133,7 +133,7 @@ static int encode_sixpack(unsigned char *, unsigned char *, int, unsigned char); static void sp_xmit_on_air(struct timer_list *t) { - struct sixpack *sp = from_timer(sp, t, tx_t); + struct sixpack *sp = timer_container_of(sp, t, tx_t); int actual, when = sp->slottime; static unsigned char random; @@ -491,7 +491,7 @@ static inline void tnc_set_sync_state(struct sixpack *sp, int new_tnc_state) static void resync_tnc(struct timer_list *t) { - struct sixpack *sp = from_timer(sp, t, resync_t); + struct sixpack *sp = timer_container_of(sp, t, resync_t); static char resync_cmd = 0xe8; /* clear any data that might have been received */ diff --git a/drivers/net/hamradio/scc.c b/drivers/net/hamradio/scc.c index f88721dec681..ae5048efde68 100644 --- a/drivers/net/hamradio/scc.c +++ b/drivers/net/hamradio/scc.c @@ -1127,7 +1127,7 @@ static inline int is_grouped(struct scc_channel *scc) static void t_dwait(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_t); + struct scc_channel *scc = timer_container_of(scc, t, tx_t); if (scc->stat.tx_state == TXS_WAIT) /* maxkeyup or idle timeout */ { @@ -1169,7 +1169,7 @@ static void t_dwait(struct timer_list *t) static void t_txdelay(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_t); + struct scc_channel *scc = timer_container_of(scc, t, tx_t); scc_start_maxkeyup(scc); @@ -1190,7 +1190,7 @@ static void t_txdelay(struct timer_list *t) static void t_tail(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_t); + struct scc_channel *scc = timer_container_of(scc, t, tx_t); unsigned long flags; spin_lock_irqsave(&scc->lock, flags); @@ -1217,7 +1217,7 @@ static void t_tail(struct timer_list *t) static void t_busy(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_wdog); + struct scc_channel *scc = timer_container_of(scc, t, tx_wdog); timer_delete(&scc->tx_t); netif_stop_queue(scc->dev); /* don't pile on the wabbit! */ @@ -1236,7 +1236,7 @@ static void t_busy(struct timer_list *t) static void t_maxkeyup(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_wdog); + struct scc_channel *scc = timer_container_of(scc, t, tx_wdog); unsigned long flags; spin_lock_irqsave(&scc->lock, flags); @@ -1270,7 +1270,7 @@ static void t_maxkeyup(struct timer_list *t) static void t_idle(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_t); + struct scc_channel *scc = timer_container_of(scc, t, tx_t); timer_delete(&scc->tx_wdog); @@ -1403,7 +1403,7 @@ static unsigned long scc_get_param(struct scc_channel *scc, unsigned int cmd) static void scc_stop_calibrate(struct timer_list *t) { - struct scc_channel *scc = from_timer(scc, t, tx_wdog); + struct scc_channel *scc = timer_container_of(scc, t, tx_wdog); unsigned long flags; spin_lock_irqsave(&scc->lock, flags); diff --git a/drivers/net/hippi/rrunner.c b/drivers/net/hippi/rrunner.c index 6342c319c0e4..7b7e7a47a75e 100644 --- a/drivers/net/hippi/rrunner.c +++ b/drivers/net/hippi/rrunner.c @@ -1154,7 +1154,7 @@ static inline void rr_raz_rx(struct rr_private *rrpriv, static void rr_timer(struct timer_list *t) { - struct rr_private *rrpriv = from_timer(rrpriv, t, timer); + struct rr_private *rrpriv = timer_container_of(rrpriv, t, timer); struct net_device *dev = pci_get_drvdata(rrpriv->pci_dev); struct rr_regs __iomem *regs = rrpriv->regs; unsigned long flags; diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 3d315e30ee47..7edbe76b5455 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -247,15 +247,39 @@ static sci_t make_sci(const u8 *addr, __be16 port) return sci; } -static sci_t macsec_frame_sci(struct macsec_eth_header *hdr, bool sci_present) +static sci_t macsec_active_sci(struct macsec_secy *secy) { - sci_t sci; + struct macsec_rx_sc *rx_sc = rcu_dereference_bh(secy->rx_sc); + + /* Case single RX SC */ + if (rx_sc && !rcu_dereference_bh(rx_sc->next)) + return (rx_sc->active) ? rx_sc->sci : 0; + /* Case no RX SC or multiple */ + else + return 0; +} + +static sci_t macsec_frame_sci(struct macsec_eth_header *hdr, bool sci_present, + struct macsec_rxh_data *rxd) +{ + struct macsec_dev *macsec; + sci_t sci = 0; - if (sci_present) + /* SC = 1 */ + if (sci_present) { memcpy(&sci, hdr->secure_channel_id, sizeof(hdr->secure_channel_id)); - else + /* SC = 0; ES = 0 */ + } else if ((!(hdr->tci_an & (MACSEC_TCI_ES | MACSEC_TCI_SC))) && + (list_is_singular(&rxd->secys))) { + /* Only one SECY should exist on this scenario */ + macsec = list_first_or_null_rcu(&rxd->secys, struct macsec_dev, + secys); + if (macsec) + return macsec_active_sci(&macsec->secy); + } else { sci = make_sci(hdr->eth.h_source, MACSEC_PORT_ES); + } return sci; } @@ -1109,7 +1133,7 @@ static rx_handler_result_t macsec_handle_frame(struct sk_buff **pskb) struct macsec_rxh_data *rxd; struct macsec_dev *macsec; unsigned int len; - sci_t sci; + sci_t sci = 0; u32 hdr_pn; bool cbit; struct pcpu_rx_sc_stats *rxsc_stats; @@ -1156,11 +1180,14 @@ static rx_handler_result_t macsec_handle_frame(struct sk_buff **pskb) macsec_skb_cb(skb)->has_sci = !!(hdr->tci_an & MACSEC_TCI_SC); macsec_skb_cb(skb)->assoc_num = hdr->tci_an & MACSEC_AN_MASK; - sci = macsec_frame_sci(hdr, macsec_skb_cb(skb)->has_sci); rcu_read_lock(); rxd = macsec_data_rcu(skb->dev); + sci = macsec_frame_sci(hdr, macsec_skb_cb(skb)->has_sci, rxd); + if (!sci) + goto drop_nosc; + list_for_each_entry_rcu(macsec, &rxd->secys, secys) { struct macsec_rx_sc *sc = find_rx_sc(&macsec->secy, sci); @@ -1283,6 +1310,7 @@ drop: macsec_rxsa_put(rx_sa); drop_nosa: macsec_rxsc_put(rx_sc); +drop_nosc: rcu_read_unlock(); drop_direct: kfree_skb(skb); diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 4289ccd3e41b..176935a8645f 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -1252,7 +1252,6 @@ static int sysdata_append_release(struct netconsole_target *nt, int offset) */ static int prepare_extradata(struct netconsole_target *nt) { - u32 fields = SYSDATA_CPU_NR | SYSDATA_TASKNAME; int extradata_len; /* userdata was appended when configfs write helper was called @@ -1260,7 +1259,7 @@ static int prepare_extradata(struct netconsole_target *nt) */ extradata_len = nt->userdata_length; - if (!(nt->sysdata_fields & fields)) + if (!nt->sysdata_fields) goto out; if (nt->sysdata_fields & SYSDATA_CPU_NR) diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c index af545d42961c..fa5fbd97ad69 100644 --- a/drivers/net/netdevsim/netdev.c +++ b/drivers/net/netdevsim/netdev.c @@ -371,7 +371,8 @@ static int nsim_poll(struct napi_struct *napi, int budget) int done; done = nsim_rcv(rq, budget); - napi_complete(napi); + if (done < budget) + napi_complete_done(napi, done); return done; } diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c index ef4204638392..fbeae05817e9 100644 --- a/drivers/net/ntb_netdev.c +++ b/drivers/net/ntb_netdev.c @@ -229,7 +229,7 @@ err: static void ntb_netdev_tx_timer(struct timer_list *t) { - struct ntb_netdev *dev = from_timer(dev, t, tx_timer); + struct ntb_netdev *dev = timer_container_of(dev, t, tx_timer); struct net_device *ndev = dev->ndev; if (ntb_transport_tx_free_entry(dev->qp) < tx_stop) { diff --git a/drivers/net/phy/mdio_bus.c b/drivers/net/phy/mdio_bus.c index a6bcb0fee863..fda2e27c1810 100644 --- a/drivers/net/phy/mdio_bus.c +++ b/drivers/net/phy/mdio_bus.c @@ -445,6 +445,9 @@ int __mdiobus_read(struct mii_bus *bus, int addr, u32 regnum) lockdep_assert_held_once(&bus->mdio_lock); + if (addr >= PHY_MAX_ADDR) + return -ENXIO; + if (bus->read) retval = bus->read(bus, addr, regnum); else @@ -474,6 +477,9 @@ int __mdiobus_write(struct mii_bus *bus, int addr, u32 regnum, u16 val) lockdep_assert_held_once(&bus->mdio_lock); + if (addr >= PHY_MAX_ADDR) + return -ENXIO; + if (bus->write) err = bus->write(bus, addr, regnum, val); else @@ -535,6 +541,9 @@ int __mdiobus_c45_read(struct mii_bus *bus, int addr, int devad, u32 regnum) lockdep_assert_held_once(&bus->mdio_lock); + if (addr >= PHY_MAX_ADDR) + return -ENXIO; + if (bus->read_c45) retval = bus->read_c45(bus, addr, devad, regnum); else @@ -566,6 +575,9 @@ int __mdiobus_c45_write(struct mii_bus *bus, int addr, int devad, u32 regnum, lockdep_assert_held_once(&bus->mdio_lock); + if (addr >= PHY_MAX_ADDR) + return -ENXIO; + if (bus->write_c45) err = bus->write_c45(bus, addr, devad, regnum, val); else diff --git a/drivers/net/phy/phy_caps.c b/drivers/net/phy/phy_caps.c index 703321689726..38417e288611 100644 --- a/drivers/net/phy/phy_caps.c +++ b/drivers/net/phy/phy_caps.c @@ -188,6 +188,9 @@ phy_caps_lookup_by_linkmode_rev(const unsigned long *linkmodes, bool fdx_only) * When @exact is not set, we return either an exact match, or matching capabilities * at lower speed, or the lowest matching speed, or NULL. * + * Non-exact matches will try to return an exact speed and duplex match, but may + * return matching capabilities with same speed but a different duplex. + * * Returns: a matched link_capabilities according to the above process, NULL * otherwise. */ @@ -195,7 +198,7 @@ const struct link_capabilities * phy_caps_lookup(int speed, unsigned int duplex, const unsigned long *supported, bool exact) { - const struct link_capabilities *lcap, *last = NULL; + const struct link_capabilities *lcap, *match = NULL, *last = NULL; for_each_link_caps_desc_speed(lcap) { if (linkmode_intersects(lcap->linkmodes, supported)) { @@ -204,16 +207,19 @@ phy_caps_lookup(int speed, unsigned int duplex, const unsigned long *supported, if (lcap->speed == speed && lcap->duplex == duplex) { return lcap; } else if (!exact) { - if (lcap->speed <= speed) - return lcap; + if (!match && lcap->speed <= speed) + match = lcap; + + if (lcap->speed < speed) + break; } } } - if (!exact) - return last; + if (!match && !exact) + match = last; - return NULL; + return match; } EXPORT_SYMBOL_GPL(phy_caps_lookup); diff --git a/drivers/net/slip/slip.c b/drivers/net/slip/slip.c index 3cfa17cd5073..c889fb374703 100644 --- a/drivers/net/slip/slip.c +++ b/drivers/net/slip/slip.c @@ -1378,7 +1378,7 @@ module_exit(slip_exit); static void sl_outfill(struct timer_list *t) { - struct slip *sl = from_timer(sl, t, outfill_timer); + struct slip *sl = timer_container_of(sl, t, outfill_timer); spin_lock(&sl->lock); @@ -1409,7 +1409,7 @@ out: static void sl_keepalive(struct timer_list *t) { - struct slip *sl = from_timer(sl, t, keepalive_timer); + struct slip *sl = timer_container_of(sl, t, keepalive_timer); spin_lock(&sl->lock); diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 1207196cbbed..f8c5e2fd04df 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -377,7 +377,7 @@ static void tun_flow_delete_by_queue(struct tun_struct *tun, u16 queue_index) static void tun_flow_cleanup(struct timer_list *t) { - struct tun_struct *tun = from_timer(tun, t, flow_gc_timer); + struct tun_struct *tun = timer_container_of(tun, t, flow_gc_timer); unsigned long delay = tun->ageing_time; unsigned long next_timer = jiffies + delay; unsigned long count = 0; diff --git a/drivers/net/usb/catc.c b/drivers/net/usb/catc.c index fc5e441aa7c3..6759388692f8 100644 --- a/drivers/net/usb/catc.c +++ b/drivers/net/usb/catc.c @@ -602,7 +602,7 @@ static void catc_stats_done(struct catc *catc, struct ctrl_queue *q) static void catc_stats_timer(struct timer_list *t) { - struct catc *catc = from_timer(catc, t, timer); + struct catc *catc = timer_container_of(catc, t, timer); int i; for (i = 0; i < 8; i++) diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c index 759dab980a09..f53e255116ea 100644 --- a/drivers/net/usb/lan78xx.c +++ b/drivers/net/usb/lan78xx.c @@ -4642,7 +4642,7 @@ static const struct net_device_ops lan78xx_netdev_ops = { static void lan78xx_stat_monitor(struct timer_list *t) { - struct lan78xx_net *dev = from_timer(dev, t, stat_monitor); + struct lan78xx_net *dev = timer_container_of(dev, t, stat_monitor); lan78xx_defer_kevent(dev, EVENT_STAT_UPDATE); } diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index d6589b24c68d..44cba7acfe7d 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -10054,6 +10054,7 @@ static const struct usb_device_id rtl8152_table[] = { { USB_DEVICE(VENDOR_ID_LINKSYS, 0x0041) }, { USB_DEVICE(VENDOR_ID_NVIDIA, 0x09ff) }, { USB_DEVICE(VENDOR_ID_TPLINK, 0x0601) }, + { USB_DEVICE(VENDOR_ID_TPLINK, 0x0602) }, { USB_DEVICE(VENDOR_ID_DLINK, 0xb301) }, { USB_DEVICE(VENDOR_ID_DELL, 0xb097) }, { USB_DEVICE(VENDOR_ID_ASUS, 0x1976) }, diff --git a/drivers/net/usb/sierra_net.c b/drivers/net/usb/sierra_net.c index dec6e82eb0e0..c30ca415d1d3 100644 --- a/drivers/net/usb/sierra_net.c +++ b/drivers/net/usb/sierra_net.c @@ -573,7 +573,7 @@ static void sierra_net_defer_kevent(struct usbnet *dev, int work) */ static void sierra_sync_timer(struct timer_list *t) { - struct sierra_net_data *priv = from_timer(priv, t, sync_timer); + struct sierra_net_data *priv = timer_container_of(priv, t, sync_timer); struct usbnet *dev = priv->usbnet; dev_dbg(&dev->udev->dev, "%s", __func__); diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c index c39dfa17813a..c04e715a4c2a 100644 --- a/drivers/net/usb/usbnet.c +++ b/drivers/net/usb/usbnet.c @@ -1551,7 +1551,7 @@ static inline void usb_free_skb(struct sk_buff *skb) static void usbnet_bh (struct timer_list *t) { - struct usbnet *dev = from_timer(dev, t, delay); + struct usbnet *dev = timer_container_of(dev, t, delay); struct sk_buff *skb; struct skb_data *entry; diff --git a/drivers/net/veth.c b/drivers/net/veth.c index e58a0f1b5c5b..a3046142cb8e 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -909,7 +909,7 @@ static int veth_xdp_rcv(struct veth_rq *rq, int budget, /* NAPI functions as RCU section */ peer_dev = rcu_dereference_check(priv->peer, rcu_read_lock_bh_held()); - peer_txq = netdev_get_tx_queue(peer_dev, queue_idx); + peer_txq = peer_dev ? netdev_get_tx_queue(peer_dev, queue_idx) : NULL; for (i = 0; i < budget; i++) { void *ptr = __ptr_ring_consume(&rq->xdp_ring); @@ -959,7 +959,7 @@ static int veth_xdp_rcv(struct veth_rq *rq, int budget, rq->stats.vs.xdp_packets += done; u64_stats_update_end(&rq->stats.syncp); - if (unlikely(netif_tx_queue_stopped(peer_txq))) + if (peer_txq && unlikely(netif_tx_queue_stopped(peer_txq))) netif_tx_wake_queue(peer_txq); return done; diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index a56d7239b127..97792de896b7 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -2809,7 +2809,7 @@ out: /* Walk the forwarding table and purge stale entries */ static void vxlan_cleanup(struct timer_list *t) { - struct vxlan_dev *vxlan = from_timer(vxlan, t, age_timer); + struct vxlan_dev *vxlan = timer_container_of(vxlan, t, age_timer); unsigned long next_timer = jiffies + FDB_AGE_INTERVAL; struct vxlan_fdb *f; diff --git a/drivers/net/wan/hdlc_cisco.c b/drivers/net/wan/hdlc_cisco.c index 7e653432c139..bfc978b15bc2 100644 --- a/drivers/net/wan/hdlc_cisco.c +++ b/drivers/net/wan/hdlc_cisco.c @@ -244,7 +244,7 @@ rx_error: static void cisco_timer(struct timer_list *t) { - struct cisco_state *st = from_timer(st, t, timer); + struct cisco_state *st = timer_container_of(st, t, timer); struct net_device *dev = st->dev; spin_lock(&st->lock); diff --git a/drivers/net/wan/hdlc_fr.c b/drivers/net/wan/hdlc_fr.c index 34014f427060..08a0ba5ca471 100644 --- a/drivers/net/wan/hdlc_fr.c +++ b/drivers/net/wan/hdlc_fr.c @@ -581,7 +581,7 @@ static void fr_set_link_state(int reliable, struct net_device *dev) static void fr_timer(struct timer_list *t) { - struct frad_state *st = from_timer(st, t, timer); + struct frad_state *st = timer_container_of(st, t, timer); struct net_device *dev = st->dev; hdlc_device *hdlc = dev_to_hdlc(dev); int i, cnt = 0, reliable; diff --git a/drivers/net/wan/hdlc_ppp.c b/drivers/net/wan/hdlc_ppp.c index 19921b02846d..7496a2e9a282 100644 --- a/drivers/net/wan/hdlc_ppp.c +++ b/drivers/net/wan/hdlc_ppp.c @@ -561,7 +561,7 @@ out: static void ppp_timer(struct timer_list *t) { - struct proto *proto = from_timer(proto, t, timer); + struct proto *proto = timer_container_of(proto, t, timer); struct ppp *ppp = get_ppp(proto->dev); unsigned long flags; diff --git a/drivers/net/wireguard/timers.c b/drivers/net/wireguard/timers.c index a9e0890c2f77..4016a3065602 100644 --- a/drivers/net/wireguard/timers.c +++ b/drivers/net/wireguard/timers.c @@ -40,8 +40,8 @@ static inline void mod_peer_timer(struct wg_peer *peer, static void wg_expired_retransmit_handshake(struct timer_list *timer) { - struct wg_peer *peer = from_timer(peer, timer, - timer_retransmit_handshake); + struct wg_peer *peer = timer_container_of(peer, timer, + timer_retransmit_handshake); if (peer->timer_handshake_attempts > MAX_TIMER_HANDSHAKES) { pr_debug("%s: Handshake for peer %llu (%pISpfsc) did not complete after %d attempts, giving up\n", @@ -78,7 +78,8 @@ static void wg_expired_retransmit_handshake(struct timer_list *timer) static void wg_expired_send_keepalive(struct timer_list *timer) { - struct wg_peer *peer = from_timer(peer, timer, timer_send_keepalive); + struct wg_peer *peer = timer_container_of(peer, timer, + timer_send_keepalive); wg_packet_send_keepalive(peer); if (peer->timer_need_another_keepalive) { @@ -90,7 +91,8 @@ static void wg_expired_send_keepalive(struct timer_list *timer) static void wg_expired_new_handshake(struct timer_list *timer) { - struct wg_peer *peer = from_timer(peer, timer, timer_new_handshake); + struct wg_peer *peer = timer_container_of(peer, timer, + timer_new_handshake); pr_debug("%s: Retrying handshake with peer %llu (%pISpfsc) because we stopped hearing back after %d seconds\n", peer->device->dev->name, peer->internal_id, @@ -104,7 +106,8 @@ static void wg_expired_new_handshake(struct timer_list *timer) static void wg_expired_zero_key_material(struct timer_list *timer) { - struct wg_peer *peer = from_timer(peer, timer, timer_zero_key_material); + struct wg_peer *peer = timer_container_of(peer, timer, + timer_zero_key_material); rcu_read_lock_bh(); if (!READ_ONCE(peer->is_dead)) { @@ -134,8 +137,8 @@ static void wg_queued_expired_zero_key_material(struct work_struct *work) static void wg_expired_send_persistent_keepalive(struct timer_list *timer) { - struct wg_peer *peer = from_timer(peer, timer, - timer_persistent_keepalive); + struct wg_peer *peer = timer_container_of(peer, timer, + timer_persistent_keepalive); if (likely(peer->persistent_keepalive_interval)) wg_packet_send_keepalive(peer); diff --git a/drivers/net/wireless/ath/ar5523/ar5523.c b/drivers/net/wireless/ath/ar5523/ar5523.c index 96dc2778022a..343c9de2749c 100644 --- a/drivers/net/wireless/ath/ar5523/ar5523.c +++ b/drivers/net/wireless/ath/ar5523/ar5523.c @@ -902,7 +902,7 @@ static void ar5523_tx_work(struct work_struct *work) static void ar5523_tx_wd_timer(struct timer_list *t) { - struct ar5523 *ar = from_timer(ar, t, tx_wd_timer); + struct ar5523 *ar = timer_container_of(ar, t, tx_wd_timer); ar5523_dbg(ar, "TX watchdog timer triggered\n"); ieee80211_queue_work(ar->hw, &ar->tx_wd_work); diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c index 52981052e211..fb0d5d4cae3a 100644 --- a/drivers/net/wireless/ath/ath10k/htt_rx.c +++ b/drivers/net/wireless/ath/ath10k/htt_rx.c @@ -257,7 +257,8 @@ static void ath10k_htt_rx_msdu_buff_replenish(struct ath10k_htt *htt) static void ath10k_htt_rx_ring_refill_retry(struct timer_list *t) { - struct ath10k_htt *htt = from_timer(htt, t, rx_ring.refill_retry_timer); + struct ath10k_htt *htt = timer_container_of(htt, t, + rx_ring.refill_retry_timer); ath10k_htt_rx_msdu_buff_replenish(htt); } diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c index 8c7ffea0fa44..07fe05384cdf 100644 --- a/drivers/net/wireless/ath/ath10k/mac.c +++ b/drivers/net/wireless/ath/ath10k/mac.c @@ -4,6 +4,7 @@ * Copyright (c) 2011-2017 Qualcomm Atheros, Inc. * Copyright (c) 2018-2019, The Linux Foundation. All rights reserved. * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. */ #include "mac.h" @@ -1022,6 +1023,26 @@ static inline int ath10k_vdev_setup_sync(struct ath10k *ar) return ar->last_wmi_vdev_start_status; } +static inline int ath10k_vdev_delete_sync(struct ath10k *ar) +{ + unsigned long time_left; + + lockdep_assert_held(&ar->conf_mutex); + + if (!test_bit(WMI_SERVICE_SYNC_DELETE_CMDS, ar->wmi.svc_map)) + return 0; + + if (test_bit(ATH10K_FLAG_CRASH_FLUSH, &ar->dev_flags)) + return -ESHUTDOWN; + + time_left = wait_for_completion_timeout(&ar->vdev_delete_done, + ATH10K_VDEV_DELETE_TIMEOUT_HZ); + if (time_left == 0) + return -ETIMEDOUT; + + return 0; +} + static int ath10k_monitor_vdev_start(struct ath10k *ar, int vdev_id) { struct cfg80211_chan_def *chandef = NULL; @@ -5900,7 +5921,6 @@ static void ath10k_remove_interface(struct ieee80211_hw *hw, struct ath10k *ar = hw->priv; struct ath10k_vif *arvif = (void *)vif->drv_priv; struct ath10k_peer *peer; - unsigned long time_left; int ret; int i; @@ -5940,13 +5960,10 @@ static void ath10k_remove_interface(struct ieee80211_hw *hw, ath10k_warn(ar, "failed to delete WMI vdev %i: %d\n", arvif->vdev_id, ret); - if (test_bit(WMI_SERVICE_SYNC_DELETE_CMDS, ar->wmi.svc_map)) { - time_left = wait_for_completion_timeout(&ar->vdev_delete_done, - ATH10K_VDEV_DELETE_TIMEOUT_HZ); - if (time_left == 0) { - ath10k_warn(ar, "Timeout in receiving vdev delete response\n"); - goto out; - } + ret = ath10k_vdev_delete_sync(ar); + if (ret) { + ath10k_warn(ar, "Error in receiving vdev delete response: %d\n", ret); + goto out; } /* Some firmware revisions don't notify host about self-peer removal diff --git a/drivers/net/wireless/ath/ath10k/pci.c b/drivers/net/wireless/ath/ath10k/pci.c index 20ec0a6d0f71..1e6d43285138 100644 --- a/drivers/net/wireless/ath/ath10k/pci.c +++ b/drivers/net/wireless/ath/ath10k/pci.c @@ -591,7 +591,7 @@ skip: static void ath10k_pci_ps_timer(struct timer_list *t) { - struct ath10k_pci *ar_pci = from_timer(ar_pci, t, ps_timer); + struct ath10k_pci *ar_pci = timer_container_of(ar_pci, t, ps_timer); struct ath10k *ar = ar_pci->ar; unsigned long flags; @@ -844,7 +844,8 @@ void ath10k_pci_rx_post(struct ath10k *ar) void ath10k_pci_rx_replenish_retry(struct timer_list *t) { - struct ath10k_pci *ar_pci = from_timer(ar_pci, t, rx_post_retry); + struct ath10k_pci *ar_pci = timer_container_of(ar_pci, t, + rx_post_retry); struct ath10k *ar = ar_pci->ar; ath10k_pci_rx_post(ar); diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c index f3212eab56a1..c06d50db40b8 100644 --- a/drivers/net/wireless/ath/ath10k/sdio.c +++ b/drivers/net/wireless/ath/ath10k/sdio.c @@ -1447,7 +1447,8 @@ release: static void ath10k_sdio_sleep_timer_handler(struct timer_list *t) { - struct ath10k_sdio *ar_sdio = from_timer(ar_sdio, t, sleep_timer); + struct ath10k_sdio *ar_sdio = timer_container_of(ar_sdio, t, + sleep_timer); ar_sdio->mbox_state = SDIO_MBOX_REQUEST_TO_SLEEP_STATE; queue_work(ar_sdio->workqueue, &ar_sdio->wr_async_work); diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c index 866bad2db334..d51f2e5a79a4 100644 --- a/drivers/net/wireless/ath/ath10k/snoc.c +++ b/drivers/net/wireless/ath/ath10k/snoc.c @@ -644,7 +644,8 @@ static void ath10k_snoc_htt_rx_cb(struct ath10k_ce_pipe *ce_state) static void ath10k_snoc_rx_replenish_retry(struct timer_list *t) { - struct ath10k_snoc *ar_snoc = from_timer(ar_snoc, t, rx_post_retry); + struct ath10k_snoc *ar_snoc = timer_container_of(ar_snoc, t, + rx_post_retry); struct ath10k *ar = ar_snoc->ar; ath10k_snoc_rx_post(ar); @@ -937,7 +938,9 @@ static int ath10k_snoc_hif_start(struct ath10k *ar) dev_set_threaded(ar->napi_dev, true); ath10k_core_napi_enable(ar); - ath10k_snoc_irq_enable(ar); + /* IRQs are left enabled when we restart due to a firmware crash */ + if (!test_bit(ATH10K_SNOC_FLAG_RECOVERY, &ar_snoc->flags)) + ath10k_snoc_irq_enable(ar); ath10k_snoc_rx_post(ar); clear_bit(ATH10K_SNOC_FLAG_RECOVERY, &ar_snoc->flags); diff --git a/drivers/net/wireless/ath/ath11k/ce.c b/drivers/net/wireless/ath/ath11k/ce.c index 9d8efec46508..746038006eb4 100644 --- a/drivers/net/wireless/ath/ath11k/ce.c +++ b/drivers/net/wireless/ath/ath11k/ce.c @@ -907,7 +907,7 @@ EXPORT_SYMBOL(ath11k_ce_rx_post_buf); void ath11k_ce_rx_replenish_retry(struct timer_list *t) { - struct ath11k_base *ab = from_timer(ab, t, rx_replenish_retry); + struct ath11k_base *ab = timer_container_of(ab, t, rx_replenish_retry); ath11k_ce_rx_post_buf(ab); } diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c index 2e9f8a5e61e4..22a101136135 100644 --- a/drivers/net/wireless/ath/ath11k/core.c +++ b/drivers/net/wireless/ath/ath11k/core.c @@ -990,6 +990,7 @@ void ath11k_fw_stats_init(struct ath11k *ar) INIT_LIST_HEAD(&ar->fw_stats.bcn); init_completion(&ar->fw_stats_complete); + init_completion(&ar->fw_stats_done); } void ath11k_fw_stats_free(struct ath11k_fw_stats *stats) @@ -2134,6 +2135,20 @@ int ath11k_core_qmi_firmware_ready(struct ath11k_base *ab) { int ret; + switch (ath11k_crypto_mode) { + case ATH11K_CRYPT_MODE_SW: + set_bit(ATH11K_FLAG_HW_CRYPTO_DISABLED, &ab->dev_flags); + set_bit(ATH11K_FLAG_RAW_MODE, &ab->dev_flags); + break; + case ATH11K_CRYPT_MODE_HW: + clear_bit(ATH11K_FLAG_HW_CRYPTO_DISABLED, &ab->dev_flags); + clear_bit(ATH11K_FLAG_RAW_MODE, &ab->dev_flags); + break; + default: + ath11k_info(ab, "invalid crypto_mode: %d\n", ath11k_crypto_mode); + return -EINVAL; + } + ret = ath11k_core_start_firmware(ab, ab->fw_mode); if (ret) { ath11k_err(ab, "failed to start firmware: %d\n", ret); @@ -2152,20 +2167,6 @@ int ath11k_core_qmi_firmware_ready(struct ath11k_base *ab) goto err_firmware_stop; } - switch (ath11k_crypto_mode) { - case ATH11K_CRYPT_MODE_SW: - set_bit(ATH11K_FLAG_HW_CRYPTO_DISABLED, &ab->dev_flags); - set_bit(ATH11K_FLAG_RAW_MODE, &ab->dev_flags); - break; - case ATH11K_CRYPT_MODE_HW: - clear_bit(ATH11K_FLAG_HW_CRYPTO_DISABLED, &ab->dev_flags); - clear_bit(ATH11K_FLAG_RAW_MODE, &ab->dev_flags); - break; - default: - ath11k_info(ab, "invalid crypto_mode: %d\n", ath11k_crypto_mode); - return -EINVAL; - } - if (ath11k_frame_mode == ATH11K_HW_TXRX_RAW) set_bit(ATH11K_FLAG_RAW_MODE, &ab->dev_flags); diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h index 339d4fca1ed5..6b2f207975e3 100644 --- a/drivers/net/wireless/ath/ath11k/core.h +++ b/drivers/net/wireless/ath/ath11k/core.h @@ -600,6 +600,8 @@ struct ath11k_fw_stats { struct list_head pdevs; struct list_head vdevs; struct list_head bcn; + u32 num_vdev_recvd; + u32 num_bcn_recvd; }; struct ath11k_dbg_htt_stats { @@ -784,7 +786,7 @@ struct ath11k { u8 alpha2[REG_ALPHA2_LEN + 1]; struct ath11k_fw_stats fw_stats; struct completion fw_stats_complete; - bool fw_stats_done; + struct completion fw_stats_done; /* protected by conf_mutex */ bool ps_state_enable; diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c index bf192529e3fe..5d46f8e4c231 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs.c +++ b/drivers/net/wireless/ath/ath11k/debugfs.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2020 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2025 Qualcomm Innovation Center, Inc. All rights reserved. */ #include <linux/vmalloc.h> @@ -93,57 +93,14 @@ void ath11k_debugfs_add_dbring_entry(struct ath11k *ar, spin_unlock_bh(&dbr_data->lock); } -static void ath11k_debugfs_fw_stats_reset(struct ath11k *ar) -{ - spin_lock_bh(&ar->data_lock); - ar->fw_stats_done = false; - ath11k_fw_stats_pdevs_free(&ar->fw_stats.pdevs); - ath11k_fw_stats_vdevs_free(&ar->fw_stats.vdevs); - spin_unlock_bh(&ar->data_lock); -} - void ath11k_debugfs_fw_stats_process(struct ath11k *ar, struct ath11k_fw_stats *stats) { struct ath11k_base *ab = ar->ab; - struct ath11k_pdev *pdev; - bool is_end; - static unsigned int num_vdev, num_bcn; - size_t total_vdevs_started = 0; - int i; - - /* WMI_REQUEST_PDEV_STAT request has been already processed */ - - if (stats->stats_id == WMI_REQUEST_RSSI_PER_CHAIN_STAT) { - ar->fw_stats_done = true; - return; - } - - if (stats->stats_id == WMI_REQUEST_VDEV_STAT) { - if (list_empty(&stats->vdevs)) { - ath11k_warn(ab, "empty vdev stats"); - return; - } - /* FW sends all the active VDEV stats irrespective of PDEV, - * hence limit until the count of all VDEVs started - */ - for (i = 0; i < ab->num_radios; i++) { - pdev = rcu_dereference(ab->pdevs_active[i]); - if (pdev && pdev->ar) - total_vdevs_started += ar->num_started_vdevs; - } - - is_end = ((++num_vdev) == total_vdevs_started); - - list_splice_tail_init(&stats->vdevs, - &ar->fw_stats.vdevs); - - if (is_end) { - ar->fw_stats_done = true; - num_vdev = 0; - } - return; - } + bool is_end = true; + /* WMI_REQUEST_PDEV_STAT, WMI_REQUEST_RSSI_PER_CHAIN_STAT and + * WMI_REQUEST_VDEV_STAT requests have been already processed. + */ if (stats->stats_id == WMI_REQUEST_BCN_STAT) { if (list_empty(&stats->bcn)) { ath11k_warn(ab, "empty bcn stats"); @@ -152,97 +109,18 @@ void ath11k_debugfs_fw_stats_process(struct ath11k *ar, struct ath11k_fw_stats * /* Mark end until we reached the count of all started VDEVs * within the PDEV */ - is_end = ((++num_bcn) == ar->num_started_vdevs); + if (ar->num_started_vdevs) + is_end = ((++ar->fw_stats.num_bcn_recvd) == + ar->num_started_vdevs); list_splice_tail_init(&stats->bcn, &ar->fw_stats.bcn); - if (is_end) { - ar->fw_stats_done = true; - num_bcn = 0; - } + if (is_end) + complete(&ar->fw_stats_done); } } -static int ath11k_debugfs_fw_stats_request(struct ath11k *ar, - struct stats_request_params *req_param) -{ - struct ath11k_base *ab = ar->ab; - unsigned long timeout, time_left; - int ret; - - lockdep_assert_held(&ar->conf_mutex); - - /* FW stats can get split when exceeding the stats data buffer limit. - * In that case, since there is no end marking for the back-to-back - * received 'update stats' event, we keep a 3 seconds timeout in case, - * fw_stats_done is not marked yet - */ - timeout = jiffies + secs_to_jiffies(3); - - ath11k_debugfs_fw_stats_reset(ar); - - reinit_completion(&ar->fw_stats_complete); - - ret = ath11k_wmi_send_stats_request_cmd(ar, req_param); - - if (ret) { - ath11k_warn(ab, "could not request fw stats (%d)\n", - ret); - return ret; - } - - time_left = wait_for_completion_timeout(&ar->fw_stats_complete, 1 * HZ); - - if (!time_left) - return -ETIMEDOUT; - - for (;;) { - if (time_after(jiffies, timeout)) - break; - - spin_lock_bh(&ar->data_lock); - if (ar->fw_stats_done) { - spin_unlock_bh(&ar->data_lock); - break; - } - spin_unlock_bh(&ar->data_lock); - } - return 0; -} - -int ath11k_debugfs_get_fw_stats(struct ath11k *ar, u32 pdev_id, - u32 vdev_id, u32 stats_id) -{ - struct ath11k_base *ab = ar->ab; - struct stats_request_params req_param; - int ret; - - mutex_lock(&ar->conf_mutex); - - if (ar->state != ATH11K_STATE_ON) { - ret = -ENETDOWN; - goto err_unlock; - } - - req_param.pdev_id = pdev_id; - req_param.vdev_id = vdev_id; - req_param.stats_id = stats_id; - - ret = ath11k_debugfs_fw_stats_request(ar, &req_param); - if (ret) - ath11k_warn(ab, "failed to request fw stats: %d\n", ret); - - ath11k_dbg(ab, ATH11K_DBG_WMI, - "debug get fw stat pdev id %d vdev id %d stats id 0x%x\n", - pdev_id, vdev_id, stats_id); - -err_unlock: - mutex_unlock(&ar->conf_mutex); - - return ret; -} - static int ath11k_open_pdev_stats(struct inode *inode, struct file *file) { struct ath11k *ar = inode->i_private; @@ -268,7 +146,7 @@ static int ath11k_open_pdev_stats(struct inode *inode, struct file *file) req_param.vdev_id = 0; req_param.stats_id = WMI_REQUEST_PDEV_STAT; - ret = ath11k_debugfs_fw_stats_request(ar, &req_param); + ret = ath11k_mac_fw_stats_request(ar, &req_param); if (ret) { ath11k_warn(ab, "failed to request fw pdev stats: %d\n", ret); goto err_free; @@ -339,7 +217,7 @@ static int ath11k_open_vdev_stats(struct inode *inode, struct file *file) req_param.vdev_id = 0; req_param.stats_id = WMI_REQUEST_VDEV_STAT; - ret = ath11k_debugfs_fw_stats_request(ar, &req_param); + ret = ath11k_mac_fw_stats_request(ar, &req_param); if (ret) { ath11k_warn(ar->ab, "failed to request fw vdev stats: %d\n", ret); goto err_free; @@ -415,7 +293,7 @@ static int ath11k_open_bcn_stats(struct inode *inode, struct file *file) continue; req_param.vdev_id = arvif->vdev_id; - ret = ath11k_debugfs_fw_stats_request(ar, &req_param); + ret = ath11k_mac_fw_stats_request(ar, &req_param); if (ret) { ath11k_warn(ar->ab, "failed to request fw bcn stats: %d\n", ret); goto err_free; diff --git a/drivers/net/wireless/ath/ath11k/debugfs.h b/drivers/net/wireless/ath/ath11k/debugfs.h index a39e458637b0..ed7fec177588 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs.h +++ b/drivers/net/wireless/ath/ath11k/debugfs.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2022, 2025 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef _ATH11K_DEBUGFS_H_ @@ -273,8 +273,6 @@ void ath11k_debugfs_unregister(struct ath11k *ar); void ath11k_debugfs_fw_stats_process(struct ath11k *ar, struct ath11k_fw_stats *stats); void ath11k_debugfs_fw_stats_init(struct ath11k *ar); -int ath11k_debugfs_get_fw_stats(struct ath11k *ar, u32 pdev_id, - u32 vdev_id, u32 stats_id); static inline bool ath11k_debugfs_is_pktlog_lite_mode_enabled(struct ath11k *ar) { @@ -381,12 +379,6 @@ static inline int ath11k_debugfs_rx_filter(struct ath11k *ar) return 0; } -static inline int ath11k_debugfs_get_fw_stats(struct ath11k *ar, - u32 pdev_id, u32 vdev_id, u32 stats_id) -{ - return 0; -} - static inline void ath11k_debugfs_add_dbring_entry(struct ath11k *ar, enum wmi_direct_buffer_module id, diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c index 3a544e5fefca..bf3928ada995 100644 --- a/drivers/net/wireless/ath/ath11k/dp.c +++ b/drivers/net/wireless/ath/ath11k/dp.c @@ -1117,8 +1117,9 @@ fail_link_desc_cleanup: static void ath11k_dp_shadow_timer_handler(struct timer_list *t) { - struct ath11k_hp_update_timer *update_timer = from_timer(update_timer, - t, timer); + struct ath11k_hp_update_timer *update_timer = timer_container_of(update_timer, + t, + timer); struct ath11k_base *ab = update_timer->ab; struct hal_srng *srng = &ab->hal.srng_list[update_timer->ring_id]; diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c index ea2959305dec..9230a965f6f0 100644 --- a/drivers/net/wireless/ath/ath11k/dp_rx.c +++ b/drivers/net/wireless/ath/ath11k/dp_rx.c @@ -308,7 +308,7 @@ static u8 *ath11k_dp_rxdesc_mpdu_start_addr2(struct ath11k_base *ab, static void ath11k_dp_service_mon_ring(struct timer_list *t) { - struct ath11k_base *ab = from_timer(ab, t, mon_reap_timer); + struct ath11k_base *ab = timer_container_of(ab, t, mon_reap_timer); int i; for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) @@ -3174,7 +3174,8 @@ move_next: static void ath11k_dp_rx_frag_timer(struct timer_list *timer) { - struct dp_rx_tid *rx_tid = from_timer(rx_tid, timer, frag_timer); + struct dp_rx_tid *rx_tid = timer_container_of(rx_tid, timer, + frag_timer); spin_lock_bh(&rx_tid->ab->base_lock); if (rx_tid->last_frag_no && diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c index 08d7b136851f..13301ca317a5 100644 --- a/drivers/net/wireless/ath/ath11k/mac.c +++ b/drivers/net/wireless/ath/ath11k/mac.c @@ -8997,6 +8997,81 @@ static void ath11k_mac_put_chain_rssi(struct station_info *sinfo, } } +static void ath11k_mac_fw_stats_reset(struct ath11k *ar) +{ + spin_lock_bh(&ar->data_lock); + ath11k_fw_stats_pdevs_free(&ar->fw_stats.pdevs); + ath11k_fw_stats_vdevs_free(&ar->fw_stats.vdevs); + ar->fw_stats.num_vdev_recvd = 0; + ar->fw_stats.num_bcn_recvd = 0; + spin_unlock_bh(&ar->data_lock); +} + +int ath11k_mac_fw_stats_request(struct ath11k *ar, + struct stats_request_params *req_param) +{ + struct ath11k_base *ab = ar->ab; + unsigned long time_left; + int ret; + + lockdep_assert_held(&ar->conf_mutex); + + ath11k_mac_fw_stats_reset(ar); + + reinit_completion(&ar->fw_stats_complete); + reinit_completion(&ar->fw_stats_done); + + ret = ath11k_wmi_send_stats_request_cmd(ar, req_param); + + if (ret) { + ath11k_warn(ab, "could not request fw stats (%d)\n", + ret); + return ret; + } + + time_left = wait_for_completion_timeout(&ar->fw_stats_complete, 1 * HZ); + if (!time_left) + return -ETIMEDOUT; + + /* FW stats can get split when exceeding the stats data buffer limit. + * In that case, since there is no end marking for the back-to-back + * received 'update stats' event, we keep a 3 seconds timeout in case, + * fw_stats_done is not marked yet + */ + time_left = wait_for_completion_timeout(&ar->fw_stats_done, 3 * HZ); + if (!time_left) + return -ETIMEDOUT; + + return 0; +} + +static int ath11k_mac_get_fw_stats(struct ath11k *ar, u32 pdev_id, + u32 vdev_id, u32 stats_id) +{ + struct ath11k_base *ab = ar->ab; + struct stats_request_params req_param; + int ret; + + lockdep_assert_held(&ar->conf_mutex); + + if (ar->state != ATH11K_STATE_ON) + return -ENETDOWN; + + req_param.pdev_id = pdev_id; + req_param.vdev_id = vdev_id; + req_param.stats_id = stats_id; + + ret = ath11k_mac_fw_stats_request(ar, &req_param); + if (ret) + ath11k_warn(ab, "failed to request fw stats: %d\n", ret); + + ath11k_dbg(ab, ATH11K_DBG_WMI, + "debug get fw stat pdev id %d vdev id %d stats id 0x%x\n", + pdev_id, vdev_id, stats_id); + + return ret; +} + static void ath11k_mac_op_sta_statistics(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_sta *sta, @@ -9031,11 +9106,12 @@ static void ath11k_mac_op_sta_statistics(struct ieee80211_hw *hw, ath11k_mac_put_chain_rssi(sinfo, arsta, "ppdu", false); + mutex_lock(&ar->conf_mutex); if (!(sinfo->filled & BIT_ULL(NL80211_STA_INFO_CHAIN_SIGNAL)) && arsta->arvif->vdev_type == WMI_VDEV_TYPE_STA && ar->ab->hw_params.supports_rssi_stats && - !ath11k_debugfs_get_fw_stats(ar, ar->pdev->pdev_id, 0, - WMI_REQUEST_RSSI_PER_CHAIN_STAT)) { + !ath11k_mac_get_fw_stats(ar, ar->pdev->pdev_id, 0, + WMI_REQUEST_RSSI_PER_CHAIN_STAT)) { ath11k_mac_put_chain_rssi(sinfo, arsta, "fw stats", true); } @@ -9043,9 +9119,10 @@ static void ath11k_mac_op_sta_statistics(struct ieee80211_hw *hw, if (!signal && arsta->arvif->vdev_type == WMI_VDEV_TYPE_STA && ar->ab->hw_params.supports_rssi_stats && - !(ath11k_debugfs_get_fw_stats(ar, ar->pdev->pdev_id, 0, - WMI_REQUEST_VDEV_STAT))) + !(ath11k_mac_get_fw_stats(ar, ar->pdev->pdev_id, 0, + WMI_REQUEST_VDEV_STAT))) signal = arsta->rssi_beacon; + mutex_unlock(&ar->conf_mutex); ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "sta statistics db2dbm %u rssi comb %d rssi beacon %d\n", @@ -9380,38 +9457,6 @@ exit: return ret; } -static int ath11k_fw_stats_request(struct ath11k *ar, - struct stats_request_params *req_param) -{ - struct ath11k_base *ab = ar->ab; - unsigned long time_left; - int ret; - - lockdep_assert_held(&ar->conf_mutex); - - spin_lock_bh(&ar->data_lock); - ar->fw_stats_done = false; - ath11k_fw_stats_pdevs_free(&ar->fw_stats.pdevs); - spin_unlock_bh(&ar->data_lock); - - reinit_completion(&ar->fw_stats_complete); - - ret = ath11k_wmi_send_stats_request_cmd(ar, req_param); - if (ret) { - ath11k_warn(ab, "could not request fw stats (%d)\n", - ret); - return ret; - } - - time_left = wait_for_completion_timeout(&ar->fw_stats_complete, - 1 * HZ); - - if (!time_left) - return -ETIMEDOUT; - - return 0; -} - static int ath11k_mac_op_get_txpower(struct ieee80211_hw *hw, struct ieee80211_vif *vif, unsigned int link_id, @@ -9419,7 +9464,6 @@ static int ath11k_mac_op_get_txpower(struct ieee80211_hw *hw, { struct ath11k *ar = hw->priv; struct ath11k_base *ab = ar->ab; - struct stats_request_params req_param = {0}; struct ath11k_fw_stats_pdev *pdev; int ret; @@ -9431,9 +9475,6 @@ static int ath11k_mac_op_get_txpower(struct ieee80211_hw *hw, */ mutex_lock(&ar->conf_mutex); - if (ar->state != ATH11K_STATE_ON) - goto err_fallback; - /* Firmware doesn't provide Tx power during CAC hence no need to fetch * the stats. */ @@ -9442,10 +9483,8 @@ static int ath11k_mac_op_get_txpower(struct ieee80211_hw *hw, return -EAGAIN; } - req_param.pdev_id = ar->pdev->pdev_id; - req_param.stats_id = WMI_REQUEST_PDEV_STAT; - - ret = ath11k_fw_stats_request(ar, &req_param); + ret = ath11k_mac_get_fw_stats(ar, ar->pdev->pdev_id, 0, + WMI_REQUEST_PDEV_STAT); if (ret) { ath11k_warn(ab, "failed to request fw pdev stats: %d\n", ret); goto err_fallback; diff --git a/drivers/net/wireless/ath/ath11k/mac.h b/drivers/net/wireless/ath/ath11k/mac.h index f5800fbecff8..5e61eea1bb03 100644 --- a/drivers/net/wireless/ath/ath11k/mac.h +++ b/drivers/net/wireless/ath/ath11k/mac.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2023, 2025 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_MAC_H @@ -179,4 +179,6 @@ int ath11k_mac_vif_set_keepalive(struct ath11k_vif *arvif, void ath11k_mac_fill_reg_tpc_info(struct ath11k *ar, struct ieee80211_vif *vif, struct ieee80211_chanctx_conf *ctx); +int ath11k_mac_fw_stats_request(struct ath11k *ar, + struct stats_request_params *req_param); #endif diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c index d7f852bebf4a..56af2e9634f4 100644 --- a/drivers/net/wireless/ath/ath11k/wmi.c +++ b/drivers/net/wireless/ath/ath11k/wmi.c @@ -8158,6 +8158,11 @@ static void ath11k_peer_assoc_conf_event(struct ath11k_base *ab, struct sk_buff static void ath11k_update_stats_event(struct ath11k_base *ab, struct sk_buff *skb) { struct ath11k_fw_stats stats = {}; + size_t total_vdevs_started = 0; + struct ath11k_pdev *pdev; + bool is_end = true; + int i; + struct ath11k *ar; int ret; @@ -8184,25 +8189,57 @@ static void ath11k_update_stats_event(struct ath11k_base *ab, struct sk_buff *sk spin_lock_bh(&ar->data_lock); - /* WMI_REQUEST_PDEV_STAT can be requested via .get_txpower mac ops or via + /* WMI_REQUEST_PDEV_STAT, WMI_REQUEST_VDEV_STAT and + * WMI_REQUEST_RSSI_PER_CHAIN_STAT can be requested via mac ops or via * debugfs fw stats. Therefore, processing it separately. */ if (stats.stats_id == WMI_REQUEST_PDEV_STAT) { list_splice_tail_init(&stats.pdevs, &ar->fw_stats.pdevs); - ar->fw_stats_done = true; + complete(&ar->fw_stats_done); + goto complete; + } + + if (stats.stats_id == WMI_REQUEST_RSSI_PER_CHAIN_STAT) { + complete(&ar->fw_stats_done); goto complete; } - /* WMI_REQUEST_VDEV_STAT, WMI_REQUEST_BCN_STAT and WMI_REQUEST_RSSI_PER_CHAIN_STAT - * are currently requested only via debugfs fw stats. Hence, processing these - * in debugfs context + if (stats.stats_id == WMI_REQUEST_VDEV_STAT) { + if (list_empty(&stats.vdevs)) { + ath11k_warn(ab, "empty vdev stats"); + goto complete; + } + /* FW sends all the active VDEV stats irrespective of PDEV, + * hence limit until the count of all VDEVs started + */ + for (i = 0; i < ab->num_radios; i++) { + pdev = rcu_dereference(ab->pdevs_active[i]); + if (pdev && pdev->ar) + total_vdevs_started += ar->num_started_vdevs; + } + + if (total_vdevs_started) + is_end = ((++ar->fw_stats.num_vdev_recvd) == + total_vdevs_started); + + list_splice_tail_init(&stats.vdevs, + &ar->fw_stats.vdevs); + + if (is_end) + complete(&ar->fw_stats_done); + + goto complete; + } + + /* WMI_REQUEST_BCN_STAT is currently requested only via debugfs fw stats. + * Hence, processing it in debugfs context */ ath11k_debugfs_fw_stats_process(ar, &stats); complete: complete(&ar->fw_stats_complete); - rcu_read_unlock(); spin_unlock_bh(&ar->data_lock); + rcu_read_unlock(); /* Since the stats's pdev, vdev and beacon list are spliced and reinitialised * at this point, no need to free the individual list. diff --git a/drivers/net/wireless/ath/ath12k/ce.c b/drivers/net/wireless/ath/ath12k/ce.c index 47821a3b060b..3f3439262cf4 100644 --- a/drivers/net/wireless/ath/ath12k/ce.c +++ b/drivers/net/wireless/ath/ath12k/ce.c @@ -868,7 +868,7 @@ void ath12k_ce_rx_post_buf(struct ath12k_base *ab) void ath12k_ce_rx_replenish_retry(struct timer_list *t) { - struct ath12k_base *ab = from_timer(ab, t, rx_replenish_retry); + struct ath12k_base *ab = timer_container_of(ab, t, rx_replenish_retry); ath12k_ce_rx_post_buf(ab); } diff --git a/drivers/net/wireless/ath/ath12k/core.c b/drivers/net/wireless/ath/ath12k/core.c index 31d851d8e688..ebc0560d40e3 100644 --- a/drivers/net/wireless/ath/ath12k/core.c +++ b/drivers/net/wireless/ath/ath12k/core.c @@ -2129,7 +2129,8 @@ int ath12k_core_init(struct ath12k_base *ab) if (!ag) { mutex_unlock(&ath12k_hw_group_mutex); ath12k_warn(ab, "unable to get hw group\n"); - return -ENODEV; + ret = -ENODEV; + goto err_unregister_notifier; } mutex_unlock(&ath12k_hw_group_mutex); @@ -2144,7 +2145,7 @@ int ath12k_core_init(struct ath12k_base *ab) if (ret) { mutex_unlock(&ag->mutex); ath12k_warn(ab, "unable to create hw group\n"); - goto err; + goto err_destroy_hw_group; } } @@ -2152,9 +2153,12 @@ int ath12k_core_init(struct ath12k_base *ab) return 0; -err: +err_destroy_hw_group: ath12k_core_hw_group_destroy(ab->ag); ath12k_core_hw_group_unassign(ab); +err_unregister_notifier: + ath12k_core_panic_notifier_unregister(ab); + return ret; } diff --git a/drivers/net/wireless/ath/ath12k/dp_rx.c b/drivers/net/wireless/ath/ath12k/dp_rx.c index 7e1a8f29c7b6..57648febc4a4 100644 --- a/drivers/net/wireless/ath/ath12k/dp_rx.c +++ b/drivers/net/wireless/ath/ath12k/dp_rx.c @@ -2955,7 +2955,8 @@ exit: static void ath12k_dp_rx_frag_timer(struct timer_list *timer) { - struct ath12k_dp_rx_tid *rx_tid = from_timer(rx_tid, timer, frag_timer); + struct ath12k_dp_rx_tid *rx_tid = timer_container_of(rx_tid, timer, + frag_timer); spin_lock_bh(&rx_tid->ab->base_lock); if (rx_tid->last_frag_no && diff --git a/drivers/net/wireless/ath/ath12k/hal.h b/drivers/net/wireless/ath/ath12k/hal.h index 0ee9c6b26dab..c1750b5dc03c 100644 --- a/drivers/net/wireless/ath/ath12k/hal.h +++ b/drivers/net/wireless/ath/ath12k/hal.h @@ -585,7 +585,8 @@ enum hal_reo_cmd_type { * or cache was blocked * @HAL_REO_CMD_FAILED: Command execution failed, could be due to * invalid queue desc - * @HAL_REO_CMD_RESOURCE_BLOCKED: + * @HAL_REO_CMD_RESOURCE_BLOCKED: Command could not be executed because + * one or more descriptors were blocked * @HAL_REO_CMD_DRAIN: */ enum hal_reo_cmd_status { diff --git a/drivers/net/wireless/ath/ath12k/hw.c b/drivers/net/wireless/ath/ath12k/hw.c index 7e2cf0fb2085..8254dc10b53b 100644 --- a/drivers/net/wireless/ath/ath12k/hw.c +++ b/drivers/net/wireless/ath/ath12k/hw.c @@ -951,6 +951,8 @@ static const struct ath12k_hw_regs qcn9274_v1_regs = { .hal_umac_ce0_dest_reg_base = 0x01b81000, .hal_umac_ce1_src_reg_base = 0x01b82000, .hal_umac_ce1_dest_reg_base = 0x01b83000, + + .gcc_gcc_pcie_hot_rst = 0x1e38338, }; static const struct ath12k_hw_regs qcn9274_v2_regs = { @@ -1042,6 +1044,8 @@ static const struct ath12k_hw_regs qcn9274_v2_regs = { .hal_umac_ce0_dest_reg_base = 0x01b81000, .hal_umac_ce1_src_reg_base = 0x01b82000, .hal_umac_ce1_dest_reg_base = 0x01b83000, + + .gcc_gcc_pcie_hot_rst = 0x1e38338, }; static const struct ath12k_hw_regs ipq5332_regs = { @@ -1215,6 +1219,8 @@ static const struct ath12k_hw_regs wcn7850_regs = { .hal_umac_ce0_dest_reg_base = 0x01b81000, .hal_umac_ce1_src_reg_base = 0x01b82000, .hal_umac_ce1_dest_reg_base = 0x01b83000, + + .gcc_gcc_pcie_hot_rst = 0x1e40304, }; static const struct ath12k_hw_hal_params ath12k_hw_hal_params_qcn9274 = { diff --git a/drivers/net/wireless/ath/ath12k/hw.h b/drivers/net/wireless/ath/ath12k/hw.h index 0fbc17649df4..0a75bc5abfa2 100644 --- a/drivers/net/wireless/ath/ath12k/hw.h +++ b/drivers/net/wireless/ath/ath12k/hw.h @@ -375,6 +375,8 @@ struct ath12k_hw_regs { u32 hal_reo_cmd_ring_base; u32 hal_reo_status_ring_base; + + u32 gcc_gcc_pcie_hot_rst; }; static inline const char *ath12k_bd_ie_type_str(enum ath12k_bd_ie_type type) diff --git a/drivers/net/wireless/ath/ath12k/pci.c b/drivers/net/wireless/ath/ath12k/pci.c index 489d546390fc..1f3cfd9b89fd 100644 --- a/drivers/net/wireless/ath/ath12k/pci.c +++ b/drivers/net/wireless/ath/ath12k/pci.c @@ -292,10 +292,10 @@ static void ath12k_pci_enable_ltssm(struct ath12k_base *ab) ath12k_dbg(ab, ATH12K_DBG_PCI, "pci ltssm 0x%x\n", val); - val = ath12k_pci_read32(ab, GCC_GCC_PCIE_HOT_RST); + val = ath12k_pci_read32(ab, GCC_GCC_PCIE_HOT_RST(ab)); val |= GCC_GCC_PCIE_HOT_RST_VAL; - ath12k_pci_write32(ab, GCC_GCC_PCIE_HOT_RST, val); - val = ath12k_pci_read32(ab, GCC_GCC_PCIE_HOT_RST); + ath12k_pci_write32(ab, GCC_GCC_PCIE_HOT_RST(ab), val); + val = ath12k_pci_read32(ab, GCC_GCC_PCIE_HOT_RST(ab)); ath12k_dbg(ab, ATH12K_DBG_PCI, "pci pcie_hot_rst 0x%x\n", val); diff --git a/drivers/net/wireless/ath/ath12k/pci.h b/drivers/net/wireless/ath/ath12k/pci.h index 0b4c459d6d8e..d1ec8aad7f6c 100644 --- a/drivers/net/wireless/ath/ath12k/pci.h +++ b/drivers/net/wireless/ath/ath12k/pci.h @@ -28,7 +28,9 @@ #define PCIE_PCIE_PARF_LTSSM 0x1e081b0 #define PARM_LTSSM_VALUE 0x111 -#define GCC_GCC_PCIE_HOT_RST 0x1e38338 +#define GCC_GCC_PCIE_HOT_RST(ab) \ + ((ab)->hw_params->regs->gcc_gcc_pcie_hot_rst) + #define GCC_GCC_PCIE_HOT_RST_VAL 0x10 #define PCIE_PCIE_INT_ALL_CLEAR 0x1e08228 diff --git a/drivers/net/wireless/ath/ath6kl/main.c b/drivers/net/wireless/ath/ath6kl/main.c index 867089a3c096..d62b96459751 100644 --- a/drivers/net/wireless/ath/ath6kl/main.c +++ b/drivers/net/wireless/ath/ath6kl/main.c @@ -503,7 +503,7 @@ void ath6kl_connect_ap_mode_sta(struct ath6kl_vif *vif, u16 aid, u8 *mac_addr, void disconnect_timer_handler(struct timer_list *t) { - struct ath6kl_vif *vif = from_timer(vif, t, disconnect_timer); + struct ath6kl_vif *vif = timer_container_of(vif, t, disconnect_timer); ath6kl_init_profile_info(vif); ath6kl_disconnect(vif); diff --git a/drivers/net/wireless/ath/ath6kl/recovery.c b/drivers/net/wireless/ath/ath6kl/recovery.c index fd2dceb8b63d..43186c193df1 100644 --- a/drivers/net/wireless/ath/ath6kl/recovery.c +++ b/drivers/net/wireless/ath/ath6kl/recovery.c @@ -62,7 +62,7 @@ void ath6kl_recovery_hb_event(struct ath6kl *ar, u32 cookie) static void ath6kl_recovery_hb_timer(struct timer_list *t) { - struct ath6kl *ar = from_timer(ar, t, fw_recovery.hb_timer); + struct ath6kl *ar = timer_container_of(ar, t, fw_recovery.hb_timer); int err; if (test_bit(RECOVERY_CLEANUP, &ar->flag) || diff --git a/drivers/net/wireless/ath/ath6kl/txrx.c b/drivers/net/wireless/ath/ath6kl/txrx.c index 3a6f0b647e17..c3b06b515c4f 100644 --- a/drivers/net/wireless/ath/ath6kl/txrx.c +++ b/drivers/net/wireless/ath/ath6kl/txrx.c @@ -1623,7 +1623,8 @@ void ath6kl_rx(struct htc_target *target, struct htc_packet *packet) static void aggr_timeout(struct timer_list *t) { u8 i, j; - struct aggr_info_conn *aggr_conn = from_timer(aggr_conn, t, timer); + struct aggr_info_conn *aggr_conn = timer_container_of(aggr_conn, t, + timer); struct rxtid *rxtid; struct rxtid_stats *stats; diff --git a/drivers/net/wireless/ath/ath6kl/wmi.c b/drivers/net/wireless/ath/ath6kl/wmi.c index 3787b9fb0075..84317afe4651 100644 --- a/drivers/net/wireless/ath/ath6kl/wmi.c +++ b/drivers/net/wireless/ath/ath6kl/wmi.c @@ -1078,7 +1078,7 @@ static int ath6kl_wmi_tkip_micerr_event_rx(struct wmi *wmi, u8 *datap, int len, void ath6kl_wmi_sscan_timer(struct timer_list *t) { - struct ath6kl_vif *vif = from_timer(vif, t, sched_scan_timer); + struct ath6kl_vif *vif = timer_container_of(vif, t, sched_scan_timer); cfg80211_sched_scan_results(vif->ar->wiphy, 0); } diff --git a/drivers/net/wireless/ath/ath9k/channel.c b/drivers/net/wireless/ath/ath9k/channel.c index 799be0be24f4..121e51ce1bc0 100644 --- a/drivers/net/wireless/ath/ath9k/channel.c +++ b/drivers/net/wireless/ath/ath9k/channel.c @@ -1040,7 +1040,7 @@ static void ath_scan_channel_start(struct ath_softc *sc) static void ath_chanctx_timer(struct timer_list *t) { - struct ath_softc *sc = from_timer(sc, t, sched.timer); + struct ath_softc *sc = timer_container_of(sc, t, sched.timer); struct ath_common *common = ath9k_hw_common(sc->sc_ah); ath_dbg(common, CHAN_CTX, @@ -1051,7 +1051,7 @@ static void ath_chanctx_timer(struct timer_list *t) static void ath_offchannel_timer(struct timer_list *t) { - struct ath_softc *sc = from_timer(sc, t, offchannel.timer); + struct ath_softc *sc = timer_container_of(sc, t, offchannel.timer); struct ath_chanctx *ctx; struct ath_common *common = ath9k_hw_common(sc->sc_ah); diff --git a/drivers/net/wireless/ath/ath9k/gpio.c b/drivers/net/wireless/ath/ath9k/gpio.c index 5a26f1d05f04..2dbc7efdd637 100644 --- a/drivers/net/wireless/ath/ath9k/gpio.c +++ b/drivers/net/wireless/ath/ath9k/gpio.c @@ -193,7 +193,7 @@ static void ath_mci_ftp_adjust(struct ath_softc *sc) */ static void ath_btcoex_period_timer(struct timer_list *t) { - struct ath_softc *sc = from_timer(sc, t, btcoex.period_timer); + struct ath_softc *sc = timer_container_of(sc, t, btcoex.period_timer); struct ath_hw *ah = sc->sc_ah; struct ath_btcoex *btcoex = &sc->btcoex; enum ath_stomp_type stomp_type; @@ -254,7 +254,8 @@ skip_hw_wakeup: */ static void ath_btcoex_no_stomp_timer(struct timer_list *t) { - struct ath_softc *sc = from_timer(sc, t, btcoex.no_stomp_timer); + struct ath_softc *sc = timer_container_of(sc, t, + btcoex.no_stomp_timer); struct ath_hw *ah = sc->sc_ah; struct ath_btcoex *btcoex = &sc->btcoex; diff --git a/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c b/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c index ce9c04e418b8..ee5945cfc10e 100644 --- a/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c +++ b/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c @@ -760,7 +760,8 @@ static void ath9k_htc_tx_cleanup_queue(struct ath9k_htc_priv *priv, void ath9k_htc_tx_cleanup_timer(struct timer_list *t) { - struct ath9k_htc_priv *priv = from_timer(priv, t, tx.cleanup_timer); + struct ath9k_htc_priv *priv = timer_container_of(priv, t, + tx.cleanup_timer); struct ath_common *common = ath9k_hw_common(priv->ah); struct ath9k_htc_tx_event *event, *tmp; struct sk_buff *skb; diff --git a/drivers/net/wireless/ath/ath9k/link.c b/drivers/net/wireless/ath/ath9k/link.c index 7f890997bb53..5d7e3ddb6dbc 100644 --- a/drivers/net/wireless/ath/ath9k/link.c +++ b/drivers/net/wireless/ath/ath9k/link.c @@ -332,7 +332,7 @@ fail_paprd: */ void ath_ani_calibrate(struct timer_list *t) { - struct ath_common *common = from_timer(common, t, ani.timer); + struct ath_common *common = timer_container_of(common, t, ani.timer); struct ath_softc *sc = common->priv; struct ath_hw *ah = sc->sc_ah; bool longcal = false; diff --git a/drivers/net/wireless/ath/ath9k/main.c b/drivers/net/wireless/ath/ath9k/main.c index 92fc5e3d756e..c56f4f3b8990 100644 --- a/drivers/net/wireless/ath/ath9k/main.c +++ b/drivers/net/wireless/ath/ath9k/main.c @@ -98,7 +98,7 @@ static bool ath9k_setpower(struct ath_softc *sc, enum ath9k_power_mode mode) void ath_ps_full_sleep(struct timer_list *t) { - struct ath_softc *sc = from_timer(sc, t, sleep_timer); + struct ath_softc *sc = timer_container_of(sc, t, sleep_timer); struct ath_common *common = ath9k_hw_common(sc->sc_ah); unsigned long flags; bool reset; diff --git a/drivers/net/wireless/ath/wcn36xx/dxe.c b/drivers/net/wireless/ath/wcn36xx/dxe.c index cc2a033e87f5..0f4df5585fd9 100644 --- a/drivers/net/wireless/ath/wcn36xx/dxe.c +++ b/drivers/net/wireless/ath/wcn36xx/dxe.c @@ -373,7 +373,7 @@ void wcn36xx_dxe_tx_ack_ind(struct wcn36xx *wcn, u32 status) static void wcn36xx_dxe_tx_timer(struct timer_list *t) { - struct wcn36xx *wcn = from_timer(wcn, t, tx_ack_timer); + struct wcn36xx *wcn = timer_container_of(wcn, t, tx_ack_timer); struct ieee80211_tx_info *info; unsigned long flags; struct sk_buff *skb; diff --git a/drivers/net/wireless/ath/wil6210/interrupt.c b/drivers/net/wireless/ath/wil6210/interrupt.c index 67172385a5d6..89d4394cedcf 100644 --- a/drivers/net/wireless/ath/wil6210/interrupt.c +++ b/drivers/net/wireless/ath/wil6210/interrupt.c @@ -179,9 +179,11 @@ void wil_mask_irq(struct wil6210_priv *wil) wil_dbg_irq(wil, "mask_irq\n"); wil6210_mask_irq_tx(wil); - wil6210_mask_irq_tx_edma(wil); + if (wil->use_enhanced_dma_hw) + wil6210_mask_irq_tx_edma(wil); wil6210_mask_irq_rx(wil); - wil6210_mask_irq_rx_edma(wil); + if (wil->use_enhanced_dma_hw) + wil6210_mask_irq_rx_edma(wil); wil6210_mask_irq_misc(wil, true); wil6210_mask_irq_pseudo(wil); } @@ -190,10 +192,12 @@ void wil_unmask_irq(struct wil6210_priv *wil) { wil_dbg_irq(wil, "unmask_irq\n"); - wil_w(wil, RGF_DMA_EP_RX_ICR + offsetof(struct RGF_ICR, ICC), - WIL_ICR_ICC_VALUE); - wil_w(wil, RGF_DMA_EP_TX_ICR + offsetof(struct RGF_ICR, ICC), - WIL_ICR_ICC_VALUE); + if (wil->use_enhanced_dma_hw) { + wil_w(wil, RGF_DMA_EP_RX_ICR + offsetof(struct RGF_ICR, ICC), + WIL_ICR_ICC_VALUE); + wil_w(wil, RGF_DMA_EP_TX_ICR + offsetof(struct RGF_ICR, ICC), + WIL_ICR_ICC_VALUE); + } wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, ICC), WIL_ICR_ICC_MISC_VALUE); wil_w(wil, RGF_INT_GEN_TX_ICR + offsetof(struct RGF_ICR, ICC), @@ -845,10 +849,12 @@ void wil6210_clear_irq(struct wil6210_priv *wil) offsetof(struct RGF_ICR, ICR)); wil_clear32(wil->csr + HOSTADDR(RGF_DMA_EP_TX_ICR) + offsetof(struct RGF_ICR, ICR)); - wil_clear32(wil->csr + HOSTADDR(RGF_INT_GEN_RX_ICR) + - offsetof(struct RGF_ICR, ICR)); - wil_clear32(wil->csr + HOSTADDR(RGF_INT_GEN_TX_ICR) + - offsetof(struct RGF_ICR, ICR)); + if (wil->use_enhanced_dma_hw) { + wil_clear32(wil->csr + HOSTADDR(RGF_INT_GEN_RX_ICR) + + offsetof(struct RGF_ICR, ICR)); + wil_clear32(wil->csr + HOSTADDR(RGF_INT_GEN_TX_ICR) + + offsetof(struct RGF_ICR, ICR)); + } wil_clear32(wil->csr + HOSTADDR(RGF_DMA_EP_MISC_ICR) + offsetof(struct RGF_ICR, ICR)); wmb(); /* make sure write completed */ diff --git a/drivers/net/wireless/ath/wil6210/netdev.c b/drivers/net/wireless/ath/wil6210/netdev.c index 59884e8e3765..55c267ab2893 100644 --- a/drivers/net/wireless/ath/wil6210/netdev.c +++ b/drivers/net/wireless/ath/wil6210/netdev.c @@ -227,7 +227,7 @@ static void wil_ndev_destructor(struct net_device *ndev) static void wil_connect_timer_fn(struct timer_list *t) { - struct wil6210_vif *vif = from_timer(vif, t, connect_timer); + struct wil6210_vif *vif = timer_container_of(vif, t, connect_timer); struct wil6210_priv *wil = vif_to_wil(vif); bool q; @@ -243,7 +243,7 @@ static void wil_connect_timer_fn(struct timer_list *t) static void wil_scan_timer_fn(struct timer_list *t) { - struct wil6210_vif *vif = from_timer(vif, t, scan_timer); + struct wil6210_vif *vif = timer_container_of(vif, t, scan_timer); struct wil6210_priv *wil = vif_to_wil(vif); clear_bit(wil_status_fwready, wil->status); @@ -253,7 +253,8 @@ static void wil_scan_timer_fn(struct timer_list *t) static void wil_p2p_discovery_timer_fn(struct timer_list *t) { - struct wil6210_vif *vif = from_timer(vif, t, p2p.discovery_timer); + struct wil6210_vif *vif = timer_container_of(vif, t, + p2p.discovery_timer); struct wil6210_priv *wil = vif_to_wil(vif); wil_dbg_misc(wil, "p2p_discovery_timer_fn\n"); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c index e0de34a3e43a..69ef8cf203d2 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c @@ -272,7 +272,8 @@ static void brcmf_btcoex_restore_part1(struct brcmf_btcoex_info *btci) */ static void brcmf_btcoex_timerfunc(struct timer_list *t) { - struct brcmf_btcoex_info *bt_local = from_timer(bt_local, t, timer); + struct brcmf_btcoex_info *bt_local = timer_container_of(bt_local, t, + timer); brcmf_dbg(TRACE, "enter\n"); bt_local->timer_on = false; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c index dc2383faddd1..b94c3619526c 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c @@ -3557,7 +3557,7 @@ static void brcmf_cfg80211_escan_timeout_worker(struct work_struct *work) static void brcmf_escan_timeout(struct timer_list *t) { struct brcmf_cfg80211_info *cfg = - from_timer(cfg, t, escan_timeout); + timer_container_of(cfg, t, escan_timeout); struct brcmf_pub *drvr = cfg->pub; if (cfg->int_escan_map || cfg->scan_request) { diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c index 9f1854b3d1a5..8f97562811d7 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c @@ -2328,7 +2328,8 @@ brcmf_pcie_fwcon_timer(struct brcmf_pciedev_info *devinfo, bool active) static void brcmf_pcie_fwcon(struct timer_list *t) { - struct brcmf_pciedev_info *devinfo = from_timer(devinfo, t, timer); + struct brcmf_pciedev_info *devinfo = timer_container_of(devinfo, t, + timer); if (!devinfo->console_active) return; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c index 93727b9a5f0d..cf26ab15ee0c 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c @@ -4121,7 +4121,7 @@ brcmf_sdio_watchdog_thread(void *data) static void brcmf_sdio_watchdog(struct timer_list *t) { - struct brcmf_sdio *bus = from_timer(bus, t, timer); + struct brcmf_sdio *bus = timer_container_of(bus, t, timer); if (bus->watchdog_tsk) { complete(&bus->watchdog_wait); diff --git a/drivers/net/wireless/intel/ipw2x00/libipw_crypto.c b/drivers/net/wireless/intel/ipw2x00/libipw_crypto.c index dfcc12aa8620..243d0c5928a2 100644 --- a/drivers/net/wireless/intel/ipw2x00/libipw_crypto.c +++ b/drivers/net/wireless/intel/ipw2x00/libipw_crypto.c @@ -110,8 +110,8 @@ static void libipw_crypt_quiescing(struct libipw_crypt_info *info) static void libipw_crypt_deinit_handler(struct timer_list *t) { - struct libipw_crypt_info *info = from_timer(info, t, - crypt_deinit_timer); + struct libipw_crypt_info *info = timer_container_of(info, t, + crypt_deinit_timer); unsigned long flags; libipw_crypt_deinit_entries(info, 0); diff --git a/drivers/net/wireless/intel/iwlegacy/3945-rs.c b/drivers/net/wireless/intel/iwlegacy/3945-rs.c index df1b8ec86651..1826c37c090c 100644 --- a/drivers/net/wireless/intel/iwlegacy/3945-rs.c +++ b/drivers/net/wireless/intel/iwlegacy/3945-rs.c @@ -168,7 +168,8 @@ il3945_rate_scale_flush_wins(struct il3945_rs_sta *rs_sta) static void il3945_bg_rate_scale_flush(struct timer_list *t) { - struct il3945_rs_sta *rs_sta = from_timer(rs_sta, t, rate_scale_flush); + struct il3945_rs_sta *rs_sta = timer_container_of(rs_sta, t, + rate_scale_flush); struct il_priv *il __maybe_unused = rs_sta->il; int unflushed = 0; unsigned long flags; diff --git a/drivers/net/wireless/intel/iwlegacy/4965-mac.c b/drivers/net/wireless/intel/iwlegacy/4965-mac.c index dc8c408902e6..8e58e97a148f 100644 --- a/drivers/net/wireless/intel/iwlegacy/4965-mac.c +++ b/drivers/net/wireless/intel/iwlegacy/4965-mac.c @@ -4051,7 +4051,7 @@ il4965_hdl_alive(struct il_priv *il, struct il_rx_buf *rxb) static void il4965_bg_stats_periodic(struct timer_list *t) { - struct il_priv *il = from_timer(il, t, stats_periodic); + struct il_priv *il = timer_container_of(il, t, stats_periodic); if (test_bit(S_EXIT_PENDING, &il->status)) return; diff --git a/drivers/net/wireless/intel/iwlegacy/common.c b/drivers/net/wireless/intel/iwlegacy/common.c index 09fb4b758704..9a86688aea67 100644 --- a/drivers/net/wireless/intel/iwlegacy/common.c +++ b/drivers/net/wireless/intel/iwlegacy/common.c @@ -4804,7 +4804,7 @@ il_check_stuck_queue(struct il_priv *il, int cnt) void il_bg_watchdog(struct timer_list *t) { - struct il_priv *il = from_timer(il, t, watchdog); + struct il_priv *il = timer_container_of(il, t, watchdog); int cnt; unsigned long timeout; diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/main.c b/drivers/net/wireless/intel/iwlwifi/dvm/main.c index 1d619384c629..dbfd45948e8b 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/main.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/main.c @@ -381,7 +381,8 @@ int iwl_send_statistics_request(struct iwl_priv *priv, u8 flags, bool clear) */ static void iwl_bg_statistics_periodic(struct timer_list *t) { - struct iwl_priv *priv = from_timer(priv, t, statistics_periodic); + struct iwl_priv *priv = timer_container_of(priv, t, + statistics_periodic); if (test_bit(STATUS_EXIT_PENDING, &priv->status)) return; @@ -537,7 +538,7 @@ static void iwl_continuous_event_trace(struct iwl_priv *priv) */ static void iwl_bg_ucode_trace(struct timer_list *t) { - struct iwl_priv *priv = from_timer(priv, t, ucode_trace); + struct iwl_priv *priv = timer_container_of(priv, t, ucode_trace); if (test_bit(STATUS_EXIT_PENDING, &priv->status)) return; diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/tt.c b/drivers/net/wireless/intel/iwlwifi/dvm/tt.c index 98f0949b3683..96831ce8da6f 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/tt.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/tt.c @@ -137,8 +137,8 @@ enum iwl_antenna_ok iwl_tx_ant_restriction(struct iwl_priv *priv) */ static void iwl_tt_check_exit_ct_kill(struct timer_list *t) { - struct iwl_priv *priv = from_timer(priv, t, - thermal_throttle.ct_kill_exit_tm); + struct iwl_priv *priv = timer_container_of(priv, t, + thermal_throttle.ct_kill_exit_tm); struct iwl_tt_mgmt *tt = &priv->thermal_throttle; if (test_bit(STATUS_EXIT_PENDING, &priv->status)) @@ -187,8 +187,8 @@ static void iwl_perform_ct_kill_task(struct iwl_priv *priv, static void iwl_tt_ready_for_ct_kill(struct timer_list *t) { - struct iwl_priv *priv = from_timer(priv, t, - thermal_throttle.ct_kill_waiting_tm); + struct iwl_priv *priv = timer_container_of(priv, t, + thermal_throttle.ct_kill_waiting_tm); struct iwl_tt_mgmt *tt = &priv->thermal_throttle; if (test_bit(STATUS_EXIT_PENDING, &priv->status)) diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c index 5c8f6dc9a3e0..5240dacf1360 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c +++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c @@ -949,7 +949,7 @@ static void iwl_dbg_tlv_apply_config(struct iwl_fw_runtime *fwrt, static void iwl_dbg_tlv_periodic_trig_handler(struct timer_list *t) { struct iwl_dbg_tlv_timer_node *timer_node = - from_timer(timer_node, t, timer); + timer_container_of(timer_node, t, timer); struct iwl_fwrt_dump_data dump_data = { .trig = (void *)timer_node->tlv->data, }; diff --git a/drivers/net/wireless/intel/iwlwifi/mld/agg.c b/drivers/net/wireless/intel/iwlwifi/mld/agg.c index bda488ae9eec..6b349270481d 100644 --- a/drivers/net/wireless/intel/iwlwifi/mld/agg.c +++ b/drivers/net/wireless/intel/iwlwifi/mld/agg.c @@ -317,7 +317,7 @@ EXPORT_SYMBOL_IF_IWLWIFI_KUNIT(iwl_mld_reorder); static void iwl_mld_rx_agg_session_expired(struct timer_list *t) { struct iwl_mld_baid_data *data = - from_timer(data, t, session_timer); + timer_container_of(data, t, session_timer); struct iwl_mld_baid_data __rcu **rcu_ptr = data->rcu_ptr; struct iwl_mld_baid_data *ba_data; struct ieee80211_link_sta *link_sta; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c index 8ec4a007b4b0..5f6797598998 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c @@ -253,7 +253,7 @@ int iwl_mvm_sta_send_to_fw(struct iwl_mvm *mvm, struct ieee80211_sta *sta, static void iwl_mvm_rx_agg_session_expired(struct timer_list *t) { struct iwl_mvm_baid_data *data = - from_timer(data, t, session_timer); + timer_container_of(data, t, session_timer); struct iwl_mvm_baid_data __rcu **rcu_ptr = data->rcu_ptr; struct iwl_mvm_baid_data *ba_data; struct ieee80211_sta *sta; diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c index 656f8b06c27b..0a9e0dbb58fb 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c @@ -1501,11 +1501,27 @@ static int _iwl_pci_resume(struct device *device, bool restore) * Scratch value was altered, this means the device was powered off, we * need to reset it completely. * Note: MAC (bits 0:7) will be cleared upon suspend even with wowlan, - * so assume that any bits there mean that the device is usable. + * but not bits [15:8]. So if we have bits set in lower word, assume + * the device is alive. + * For older devices, just try silently to grab the NIC. */ - if (trans->mac_cfg->device_family >= IWL_DEVICE_FAMILY_BZ && - !iwl_read32(trans, CSR_FUNC_SCRATCH)) - device_was_powered_off = true; + if (trans->mac_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) { + if (!(iwl_read32(trans, CSR_FUNC_SCRATCH) & + CSR_FUNC_SCRATCH_POWER_OFF_MASK)) + device_was_powered_off = true; + } else { + /* + * bh are re-enabled by iwl_trans_pcie_release_nic_access, + * so re-enable them if _iwl_trans_pcie_grab_nic_access fails. + */ + local_bh_disable(); + if (_iwl_trans_pcie_grab_nic_access(trans, true)) { + iwl_trans_pcie_release_nic_access(trans); + } else { + device_was_powered_off = true; + local_bh_enable(); + } + } if (restore || device_was_powered_off) { trans->state = IWL_TRANS_NO_FW; diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c index 7abd7c7daa89..bb467e2b1779 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c @@ -700,7 +700,7 @@ void iwl_txq_log_scd_error(struct iwl_trans *trans, struct iwl_txq *txq) static void iwl_txq_stuck_timer(struct timer_list *t) { - struct iwl_txq *txq = from_timer(txq, t, stuck_timer); + struct iwl_txq *txq = timer_container_of(txq, t, stuck_timer); struct iwl_trans *trans = txq->trans; spin_lock(&txq->lock); diff --git a/drivers/net/wireless/marvell/libertas/if_usb.c b/drivers/net/wireless/marvell/libertas/if_usb.c index ea3cc2eaec36..b3c4040257a6 100644 --- a/drivers/net/wireless/marvell/libertas/if_usb.c +++ b/drivers/net/wireless/marvell/libertas/if_usb.c @@ -165,7 +165,7 @@ static void if_usb_setup_firmware(struct lbs_private *priv) static void if_usb_fw_timeo(struct timer_list *t) { - struct if_usb_card *cardp = from_timer(cardp, t, fw_timeout); + struct if_usb_card *cardp = timer_container_of(cardp, t, fw_timeout); if (cardp->fwdnldover) { lbs_deb_usb("Download complete, no event. Assuming success\n"); diff --git a/drivers/net/wireless/marvell/libertas/main.c b/drivers/net/wireless/marvell/libertas/main.c index 26d13e9b3c95..d44e02c6fe38 100644 --- a/drivers/net/wireless/marvell/libertas/main.c +++ b/drivers/net/wireless/marvell/libertas/main.c @@ -693,7 +693,7 @@ EXPORT_SYMBOL_GPL(lbs_resume); */ static void lbs_cmd_timeout_handler(struct timer_list *t) { - struct lbs_private *priv = from_timer(priv, t, command_timer); + struct lbs_private *priv = timer_container_of(priv, t, command_timer); unsigned long flags; spin_lock_irqsave(&priv->driver_lock, flags); @@ -727,7 +727,8 @@ out: */ static void lbs_tx_lockup_handler(struct timer_list *t) { - struct lbs_private *priv = from_timer(priv, t, tx_lockup_timer); + struct lbs_private *priv = timer_container_of(priv, t, + tx_lockup_timer); unsigned long flags; spin_lock_irqsave(&priv->driver_lock, flags); diff --git a/drivers/net/wireless/marvell/libertas_tf/if_usb.c b/drivers/net/wireless/marvell/libertas_tf/if_usb.c index 7c413dc81f9a..5662a244f82a 100644 --- a/drivers/net/wireless/marvell/libertas_tf/if_usb.c +++ b/drivers/net/wireless/marvell/libertas_tf/if_usb.c @@ -113,7 +113,7 @@ static void if_usb_setup_firmware(struct lbtf_private *priv) static void if_usb_fw_timeo(struct timer_list *t) { - struct if_usb_card *cardp = from_timer(cardp, t, fw_timeout); + struct if_usb_card *cardp = timer_container_of(cardp, t, fw_timeout); lbtf_deb_enter(LBTF_DEB_USB); if (!cardp->fwdnldover) { diff --git a/drivers/net/wireless/marvell/libertas_tf/main.c b/drivers/net/wireless/marvell/libertas_tf/main.c index a57a11be57d8..50c0f6179e2d 100644 --- a/drivers/net/wireless/marvell/libertas_tf/main.c +++ b/drivers/net/wireless/marvell/libertas_tf/main.c @@ -126,7 +126,7 @@ static void lbtf_cmd_work(struct work_struct *work) */ static void command_timer_fn(struct timer_list *t) { - struct lbtf_private *priv = from_timer(priv, t, command_timer); + struct lbtf_private *priv = timer_container_of(priv, t, command_timer); unsigned long flags; lbtf_deb_enter(LBTF_DEB_CMD); diff --git a/drivers/net/wireless/marvell/mwifiex/11n.c b/drivers/net/wireless/marvell/mwifiex/11n.c index 738bafc3749b..66f0f5377ac1 100644 --- a/drivers/net/wireless/marvell/mwifiex/11n.c +++ b/drivers/net/wireless/marvell/mwifiex/11n.c @@ -403,14 +403,12 @@ mwifiex_cmd_append_11n_tlv(struct mwifiex_private *priv, if (sband->ht_cap.cap & IEEE80211_HT_CAP_SUP_WIDTH_20_40 && bss_desc->bcn_ht_oper->ht_param & - IEEE80211_HT_PARAM_CHAN_WIDTH_ANY) { - chan_list->chan_scan_param[0].radio_type |= - CHAN_BW_40MHZ << 2; + IEEE80211_HT_PARAM_CHAN_WIDTH_ANY) SET_SECONDARYCHAN(chan_list->chan_scan_param[0]. radio_type, (bss_desc->bcn_ht_oper->ht_param & IEEE80211_HT_PARAM_CHA_SEC_OFFSET)); - } + *buffer += struct_size(chan_list, chan_scan_param, 1); ret_len += struct_size(chan_list, chan_scan_param, 1); } diff --git a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c index 8aff1df09b40..354c5ce66045 100644 --- a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c +++ b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c @@ -300,7 +300,7 @@ static void mwifiex_flush_data(struct timer_list *t) { struct reorder_tmr_cnxt *ctx = - from_timer(ctx, t, timer); + timer_container_of(ctx, t, timer); int start_win, seq_num; ctx->timer_is_set = false; diff --git a/drivers/net/wireless/marvell/mwifiex/cmdevt.c b/drivers/net/wireless/marvell/mwifiex/cmdevt.c index 3bf27efe4537..0f466c31337f 100644 --- a/drivers/net/wireless/marvell/mwifiex/cmdevt.c +++ b/drivers/net/wireless/marvell/mwifiex/cmdevt.c @@ -934,7 +934,8 @@ void mwifiex_process_assoc_resp(struct mwifiex_adapter *adapter) void mwifiex_cmd_timeout_func(struct timer_list *t) { - struct mwifiex_adapter *adapter = from_timer(adapter, t, cmd_timer); + struct mwifiex_adapter *adapter = timer_container_of(adapter, t, + cmd_timer); struct cmd_ctrl_node *cmd_node; set_bit(MWIFIEX_IS_CMD_TIMEDOUT, &adapter->work_flags); diff --git a/drivers/net/wireless/marvell/mwifiex/init.c b/drivers/net/wireless/marvell/mwifiex/init.c index 32c374e47794..4820010a86f6 100644 --- a/drivers/net/wireless/marvell/mwifiex/init.c +++ b/drivers/net/wireless/marvell/mwifiex/init.c @@ -41,7 +41,8 @@ static int mwifiex_add_bss_prio_tbl(struct mwifiex_private *priv) static void wakeup_timer_fn(struct timer_list *t) { - struct mwifiex_adapter *adapter = from_timer(adapter, t, wakeup_timer); + struct mwifiex_adapter *adapter = timer_container_of(adapter, t, + wakeup_timer); mwifiex_dbg(adapter, ERROR, "Firmware wakeup failed\n"); adapter->hw_status = MWIFIEX_HW_STATUS_RESET; diff --git a/drivers/net/wireless/marvell/mwifiex/tdls.c b/drivers/net/wireless/marvell/mwifiex/tdls.c index 18e8c04d14c4..77a9a6de636d 100644 --- a/drivers/net/wireless/marvell/mwifiex/tdls.c +++ b/drivers/net/wireless/marvell/mwifiex/tdls.c @@ -1415,7 +1415,8 @@ void mwifiex_auto_tdls_update_peer_signal(struct mwifiex_private *priv, void mwifiex_check_auto_tdls(struct timer_list *t) { - struct mwifiex_private *priv = from_timer(priv, t, auto_tdls_timer); + struct mwifiex_private *priv = timer_container_of(priv, t, + auto_tdls_timer); struct mwifiex_auto_tdls_peer *tdls_peer; u16 reason = WLAN_REASON_TDLS_TEARDOWN_UNSPECIFIED; diff --git a/drivers/net/wireless/marvell/mwifiex/usb.c b/drivers/net/wireless/marvell/mwifiex/usb.c index 2f565397cf36..947ecb0a7b40 100644 --- a/drivers/net/wireless/marvell/mwifiex/usb.c +++ b/drivers/net/wireless/marvell/mwifiex/usb.c @@ -1125,7 +1125,7 @@ static void mwifiex_usb_tx_aggr_tmo(struct timer_list *t) struct urb_context *urb_cnxt = NULL; struct sk_buff *skb_send = NULL; struct tx_aggr_tmr_cnxt *timer_context = - from_timer(timer_context, t, hold_timer); + timer_container_of(timer_context, t, hold_timer); struct mwifiex_adapter *adapter = timer_context->adapter; struct usb_tx_data_port *port = timer_context->port; int err = 0; diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c index c54005df08ca..8a37fb37f77d 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c @@ -1043,7 +1043,7 @@ void mt7615_roc_work(struct work_struct *work) void mt7615_roc_timer(struct timer_list *timer) { - struct mt7615_phy *phy = from_timer(phy, timer, roc_timer); + struct mt7615_phy *phy = timer_container_of(phy, timer, roc_timer); ieee80211_queue_work(phy->mt76->hw, &phy->roc_work); } diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c index 38dd58f6e493..a50c1723ca29 100644 --- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c +++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c @@ -283,7 +283,7 @@ EXPORT_SYMBOL_GPL(mt792x_tx_worker); void mt792x_roc_timer(struct timer_list *timer) { - struct mt792x_phy *phy = from_timer(phy, timer, roc_timer); + struct mt792x_phy *phy = timer_container_of(phy, timer, roc_timer); ieee80211_queue_work(phy->mt76->hw, &phy->roc_work); } @@ -291,7 +291,7 @@ EXPORT_SYMBOL_GPL(mt792x_roc_timer); void mt792x_csa_timer(struct timer_list *timer) { - struct mt792x_vif *mvif = from_timer(mvif, timer, csa_timer); + struct mt792x_vif *mvif = timer_container_of(mvif, timer, csa_timer); ieee80211_queue_work(mvif->phy->mt76->hw, &mvif->csa_work); } diff --git a/drivers/net/wireless/microchip/wilc1000/hif.c b/drivers/net/wireless/microchip/wilc1000/hif.c index cb46a39ef757..a229c6cab332 100644 --- a/drivers/net/wireless/microchip/wilc1000/hif.c +++ b/drivers/net/wireless/microchip/wilc1000/hif.c @@ -953,8 +953,8 @@ static void wilc_handle_listen_state_expired(struct work_struct *work) static void listen_timer_cb(struct timer_list *t) { - struct host_if_drv *hif_drv = from_timer(hif_drv, t, - remain_on_ch_timer); + struct host_if_drv *hif_drv = timer_container_of(hif_drv, t, + remain_on_ch_timer); struct wilc_vif *vif = hif_drv->remain_on_ch_timer_vif; int result; struct host_if_msg *msg; @@ -1075,7 +1075,8 @@ static void handle_scan_complete(struct work_struct *work) static void timer_scan_cb(struct timer_list *t) { - struct host_if_drv *hif_drv = from_timer(hif_drv, t, scan_timer); + struct host_if_drv *hif_drv = timer_container_of(hif_drv, t, + scan_timer); struct wilc_vif *vif = hif_drv->scan_timer_vif; struct host_if_msg *msg; int result; @@ -1091,8 +1092,8 @@ static void timer_scan_cb(struct timer_list *t) static void timer_connect_cb(struct timer_list *t) { - struct host_if_drv *hif_drv = from_timer(hif_drv, t, - connect_timer); + struct host_if_drv *hif_drv = timer_container_of(hif_drv, t, + connect_timer); struct wilc_vif *vif = hif_drv->connect_timer_vif; struct host_if_msg *msg; int result; @@ -1497,7 +1498,7 @@ int wilc_hif_set_cfg(struct wilc_vif *vif, struct cfg_param_attr *param) static void get_periodic_rssi(struct timer_list *t) { - struct wilc_vif *vif = from_timer(vif, t, periodic_rssi); + struct wilc_vif *vif = timer_container_of(vif, t, periodic_rssi); if (!vif->hif_drv) { netdev_err(vif->ndev, "%s: hif driver is NULL", __func__); diff --git a/drivers/net/wireless/purelifi/plfxlc/usb.c b/drivers/net/wireless/purelifi/plfxlc/usb.c index c2a1234b59db..d8b0b79dea1a 100644 --- a/drivers/net/wireless/purelifi/plfxlc/usb.c +++ b/drivers/net/wireless/purelifi/plfxlc/usb.c @@ -548,7 +548,7 @@ error: static void slif_data_plane_sap_timer_callb(struct timer_list *t) { - struct plfxlc_usb *usb = from_timer(usb, t, tx.tx_retry_timer); + struct plfxlc_usb *usb = timer_container_of(usb, t, tx.tx_retry_timer); plfxlc_send_packet_from_data_queue(usb); timer_setup(&usb->tx.tx_retry_timer, @@ -558,7 +558,7 @@ static void slif_data_plane_sap_timer_callb(struct timer_list *t) static void sta_queue_cleanup_timer_callb(struct timer_list *t) { - struct plfxlc_usb *usb = from_timer(usb, t, sta_queue_cleanup); + struct plfxlc_usb *usb = timer_container_of(usb, t, sta_queue_cleanup); struct plfxlc_usb_tx *tx = &usb->tx; int sidx; diff --git a/drivers/net/wireless/realtek/rtlwifi/base.c b/drivers/net/wireless/realtek/rtlwifi/base.c index 6189edc1d8d7..e26feb8de658 100644 --- a/drivers/net/wireless/realtek/rtlwifi/base.c +++ b/drivers/net/wireless/realtek/rtlwifi/base.c @@ -2220,7 +2220,8 @@ label_lps_done: void rtl_watch_dog_timer_callback(struct timer_list *t) { - struct rtl_priv *rtlpriv = from_timer(rtlpriv, t, works.watchdog_timer); + struct rtl_priv *rtlpriv = timer_container_of(rtlpriv, t, + works.watchdog_timer); queue_delayed_work(rtlpriv->works.rtl_wq, &rtlpriv->works.watchdog_wq, 0); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/dm.c b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/dm.c index 5a34894a533b..f749d19ec5f5 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/dm.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/dm.c @@ -1684,7 +1684,8 @@ static void rtl88e_dm_fast_ant_training(struct ieee80211_hw *hw) void rtl88e_dm_fast_antenna_training_callback(struct timer_list *t) { struct rtl_priv *rtlpriv = - from_timer(rtlpriv, t, works.fast_antenna_training_timer); + timer_container_of(rtlpriv, t, + works.fast_antenna_training_timer); struct ieee80211_hw *hw = rtlpriv->hw; rtl88e_dm_fast_ant_training(hw); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/hw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/hw.c index 27f6c35ba0f9..d122f1eb345e 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/hw.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/hw.c @@ -237,8 +237,8 @@ static void _rtl88ee_set_fw_ps_rf_off_low_power(struct ieee80211_hw *hw) void rtl88ee_fw_clk_off_timer_callback(struct timer_list *t) { - struct rtl_priv *rtlpriv = from_timer(rtlpriv, t, - works.fw_clockoff_timer); + struct rtl_priv *rtlpriv = timer_container_of(rtlpriv, t, + works.fw_clockoff_timer); struct ieee80211_hw *hw = rtlpriv->hw; _rtl88ee_set_fw_ps_rf_off_low_power(hw); diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c index 6ed470dd6f22..2ab440cb2d67 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.c +++ b/drivers/net/wireless/realtek/rtw88/tx.c @@ -178,7 +178,8 @@ static void rtw_tx_report_enable(struct rtw_dev *rtwdev, void rtw_tx_report_purge_timer(struct timer_list *t) { - struct rtw_dev *rtwdev = from_timer(rtwdev, t, tx_report.purge_timer); + struct rtw_dev *rtwdev = timer_container_of(rtwdev, t, + tx_report.purge_timer); struct rtw_tx_report *tx_report = &rtwdev->tx_report; unsigned long flags; diff --git a/drivers/net/wireless/rsi/rsi_91x_hal.c b/drivers/net/wireless/rsi/rsi_91x_hal.c index 53827657abb2..7d26314a3e76 100644 --- a/drivers/net/wireless/rsi/rsi_91x_hal.c +++ b/drivers/net/wireless/rsi/rsi_91x_hal.c @@ -490,7 +490,7 @@ int rsi_prepare_beacon(struct rsi_common *common, struct sk_buff *skb) static void bl_cmd_timeout(struct timer_list *t) { - struct rsi_hw *adapter = from_timer(adapter, t, bl_cmd_timer); + struct rsi_hw *adapter = timer_container_of(adapter, t, bl_cmd_timer); adapter->blcmd_timer_expired = true; timer_delete(&adapter->bl_cmd_timer); diff --git a/drivers/net/wireless/rsi/rsi_91x_mac80211.c b/drivers/net/wireless/rsi/rsi_91x_mac80211.c index 9db08200f4fa..0e115b428f96 100644 --- a/drivers/net/wireless/rsi/rsi_91x_mac80211.c +++ b/drivers/net/wireless/rsi/rsi_91x_mac80211.c @@ -1746,7 +1746,7 @@ static void rsi_resume_conn_channel(struct rsi_common *common) void rsi_roc_timeout(struct timer_list *t) { - struct rsi_common *common = from_timer(common, t, roc_timer); + struct rsi_common *common = timer_container_of(common, t, roc_timer); rsi_dbg(INFO_ZONE, "Remain on channel expired\n"); diff --git a/drivers/net/wireless/st/cw1200/queue.c b/drivers/net/wireless/st/cw1200/queue.c index 4fd76183c368..a933e2c7dc2c 100644 --- a/drivers/net/wireless/st/cw1200/queue.c +++ b/drivers/net/wireless/st/cw1200/queue.c @@ -133,7 +133,7 @@ static void cw1200_queue_gc(struct timer_list *t) { LIST_HEAD(list); struct cw1200_queue *queue = - from_timer(queue, t, gc); + timer_container_of(queue, t, gc); spin_lock_bh(&queue->lock); __cw1200_queue_gc(queue, &list, true); diff --git a/drivers/net/wireless/st/cw1200/sta.c b/drivers/net/wireless/st/cw1200/sta.c index 444272caf124..5dd7f6a38900 100644 --- a/drivers/net/wireless/st/cw1200/sta.c +++ b/drivers/net/wireless/st/cw1200/sta.c @@ -2112,7 +2112,8 @@ void cw1200_multicast_stop_work(struct work_struct *work) void cw1200_mcast_timeout(struct timer_list *t) { - struct cw1200_common *priv = from_timer(priv, t, mcast_timeout); + struct cw1200_common *priv = timer_container_of(priv, t, + mcast_timeout); wiphy_warn(priv->hw->wiphy, "Multicast delivery timeout.\n"); diff --git a/drivers/net/wireless/ti/wlcore/main.c b/drivers/net/wireless/ti/wlcore/main.c index ea9bc4717a85..f93c95edd991 100644 --- a/drivers/net/wireless/ti/wlcore/main.c +++ b/drivers/net/wireless/ti/wlcore/main.c @@ -189,7 +189,8 @@ out: static void wl1271_rx_streaming_timer(struct timer_list *t) { - struct wl12xx_vif *wlvif = from_timer(wlvif, t, rx_streaming_timer); + struct wl12xx_vif *wlvif = timer_container_of(wlvif, t, + rx_streaming_timer); struct wl1271 *wl = wlvif->wl; ieee80211_queue_work(wl->hw, &wlvif->rx_streaming_disable_work); } diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c index 5836995d6774..c759ebc56457 100644 --- a/drivers/net/xen-netback/netback.c +++ b/drivers/net/xen-netback/netback.c @@ -198,7 +198,8 @@ static void tx_add_credit(struct xenvif_queue *queue) void xenvif_tx_credit_callback(struct timer_list *t) { - struct xenvif_queue *queue = from_timer(queue, t, credit_timeout); + struct xenvif_queue *queue = timer_container_of(queue, t, + credit_timeout); tx_add_credit(queue); xenvif_napi_schedule_or_enable_events(queue); } diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 5091e1fa4a0d..9bac50963477 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -245,7 +245,8 @@ static bool xennet_can_sg(struct net_device *dev) static void rx_refill_timeout(struct timer_list *t) { - struct netfront_queue *queue = from_timer(queue, t, rx_refill_timer); + struct netfront_queue *queue = timer_container_of(queue, t, + rx_refill_timer); napi_schedule(&queue->napi); } diff --git a/drivers/nfc/nfcmrvl/fw_dnld.c b/drivers/nfc/nfcmrvl/fw_dnld.c index 43ce0c9b2355..a9b03dcc4100 100644 --- a/drivers/nfc/nfcmrvl/fw_dnld.c +++ b/drivers/nfc/nfcmrvl/fw_dnld.c @@ -119,7 +119,8 @@ static void fw_dnld_over(struct nfcmrvl_private *priv, u32 error) static void fw_dnld_timeout(struct timer_list *t) { - struct nfcmrvl_private *priv = from_timer(priv, t, fw_dnld.timer); + struct nfcmrvl_private *priv = timer_container_of(priv, t, + fw_dnld.timer); nfc_err(priv->dev, "FW loading timeout"); priv->fw_dnld.state = STATE_RESET; diff --git a/drivers/nfc/pn533/pn533.c b/drivers/nfc/pn533/pn533.c index 34c40d10e260..14661249c690 100644 --- a/drivers/nfc/pn533/pn533.c +++ b/drivers/nfc/pn533/pn533.c @@ -1233,7 +1233,7 @@ static int pn533_init_target_complete(struct pn533 *dev, struct sk_buff *resp) static void pn533_listen_mode_timer(struct timer_list *t) { - struct pn533 *dev = from_timer(dev, t, listen_timer); + struct pn533 *dev = timer_container_of(dev, t, listen_timer); dev->cancel_listen = 1; diff --git a/drivers/nfc/pn533/uart.c b/drivers/nfc/pn533/uart.c index 580c9193e4a7..a081bce61c29 100644 --- a/drivers/nfc/pn533/uart.c +++ b/drivers/nfc/pn533/uart.c @@ -133,7 +133,7 @@ static const struct pn533_phy_ops uart_phy_ops = { static void pn532_cmd_timeout(struct timer_list *t) { - struct pn532_uart_phy *dev = from_timer(dev, t, cmd_timeout); + struct pn532_uart_phy *dev = timer_container_of(dev, t, cmd_timeout); pn532_uart_send_frame(dev->priv, dev->cur_out_buf); } diff --git a/drivers/nfc/st-nci/ndlc.c b/drivers/nfc/st-nci/ndlc.c index 8feac119a4bc..be4808859cfa 100644 --- a/drivers/nfc/st-nci/ndlc.c +++ b/drivers/nfc/st-nci/ndlc.c @@ -237,14 +237,14 @@ EXPORT_SYMBOL(ndlc_recv); static void ndlc_t1_timeout(struct timer_list *t) { - struct llt_ndlc *ndlc = from_timer(ndlc, t, t1_timer); + struct llt_ndlc *ndlc = timer_container_of(ndlc, t, t1_timer); schedule_work(&ndlc->sm_work); } static void ndlc_t2_timeout(struct timer_list *t) { - struct llt_ndlc *ndlc = from_timer(ndlc, t, t2_timer); + struct llt_ndlc *ndlc = timer_container_of(ndlc, t, t2_timer); schedule_work(&ndlc->sm_work); } diff --git a/drivers/nfc/st-nci/se.c b/drivers/nfc/st-nci/se.c index 8cfe5405bae6..607ec768eb7b 100644 --- a/drivers/nfc/st-nci/se.c +++ b/drivers/nfc/st-nci/se.c @@ -696,7 +696,8 @@ static void st_nci_se_wt_timeout(struct timer_list *t) */ /* hardware reset managed through VCC_UICC_OUT power supply */ u8 param = 0x01; - struct st_nci_info *info = from_timer(info, t, se_info.bwi_timer); + struct st_nci_info *info = timer_container_of(info, t, + se_info.bwi_timer); info->se_info.bwi_active = false; @@ -714,8 +715,8 @@ static void st_nci_se_wt_timeout(struct timer_list *t) static void st_nci_se_activation_timeout(struct timer_list *t) { - struct st_nci_info *info = from_timer(info, t, - se_info.se_active_timer); + struct st_nci_info *info = timer_container_of(info, t, + se_info.se_active_timer); info->se_info.se_active = false; diff --git a/drivers/nfc/st21nfca/se.c b/drivers/nfc/st21nfca/se.c index 9a50f3c03bd4..7154bc1d644a 100644 --- a/drivers/nfc/st21nfca/se.c +++ b/drivers/nfc/st21nfca/se.c @@ -280,15 +280,16 @@ static void st21nfca_se_wt_work(struct work_struct *work) static void st21nfca_se_wt_timeout(struct timer_list *t) { - struct st21nfca_hci_info *info = from_timer(info, t, se_info.bwi_timer); + struct st21nfca_hci_info *info = timer_container_of(info, t, + se_info.bwi_timer); schedule_work(&info->se_info.timeout_work); } static void st21nfca_se_activation_timeout(struct timer_list *t) { - struct st21nfca_hci_info *info = from_timer(info, t, - se_info.se_active_timer); + struct st21nfca_hci_info *info = timer_container_of(info, t, + se_info.se_active_timer); info->se_info.se_active = false; diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 140079ff86e6..e040e467f9fa 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -998,7 +998,7 @@ void nvme_mpath_update(struct nvme_ctrl *ctrl) static void nvme_anatt_timeout(struct timer_list *t) { - struct nvme_ctrl *ctrl = from_timer(ctrl, t, anatt_timer); + struct nvme_ctrl *ctrl = timer_container_of(ctrl, t, anatt_timer); dev_info(ctrl->device, "ANATT timeout, resetting controller.\n"); nvme_reset_ctrl(ctrl); diff --git a/drivers/parport/ieee1284.c b/drivers/parport/ieee1284.c index 474515d27e9c..4035010249cd 100644 --- a/drivers/parport/ieee1284.c +++ b/drivers/parport/ieee1284.c @@ -40,7 +40,7 @@ static void parport_ieee1284_wakeup (struct parport *port) static void timeout_waiting_on_port (struct timer_list *t) { - struct parport *port = from_timer(port, t, timer); + struct parport *port = timer_container_of(port, t, timer); parport_ieee1284_wakeup (port); } diff --git a/drivers/pci/hotplug/cpqphp_ctrl.c b/drivers/pci/hotplug/cpqphp_ctrl.c index 20529d1a3c44..760a5dec0431 100644 --- a/drivers/pci/hotplug/cpqphp_ctrl.c +++ b/drivers/pci/hotplug/cpqphp_ctrl.c @@ -1883,7 +1883,7 @@ void cpqhp_pushbutton_thread(struct timer_list *t) { u8 hp_slot; struct pci_func *func; - struct slot *p_slot = from_timer(p_slot, t, task_event); + struct slot *p_slot = timer_container_of(p_slot, t, task_event); struct controller *ctrl = (struct controller *) p_slot->ctrl; pushbutton_pending = NULL; diff --git a/drivers/pci/hotplug/shpchp_hpc.c b/drivers/pci/hotplug/shpchp_hpc.c index 387b85585263..183bf43510a1 100644 --- a/drivers/pci/hotplug/shpchp_hpc.c +++ b/drivers/pci/hotplug/shpchp_hpc.c @@ -211,7 +211,7 @@ static inline int shpc_indirect_read(struct controller *ctrl, int index, */ static void int_poll_timeout(struct timer_list *t) { - struct controller *ctrl = from_timer(ctrl, t, poll_timer); + struct controller *ctrl = timer_container_of(ctrl, t, poll_timer); /* Poll for interrupt events. regs == NULL => polling */ shpc_isr(0, ctrl); diff --git a/drivers/pci/msi/irqdomain.c b/drivers/pci/msi/irqdomain.c index d7ba8795d60f..c05152733993 100644 --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -271,6 +271,7 @@ static bool pci_create_device_domain(struct pci_dev *pdev, const struct msi_doma /** * pci_setup_msi_device_domain - Setup a device MSI interrupt domain * @pdev: The PCI device to create the domain on + * @hwsize: The maximum number of MSI vectors * * Return: * True when: @@ -287,7 +288,7 @@ static bool pci_create_device_domain(struct pci_dev *pdev, const struct msi_doma * - The device is removed * - MSI is disabled and a MSI-X domain is created */ -bool pci_setup_msi_device_domain(struct pci_dev *pdev) +bool pci_setup_msi_device_domain(struct pci_dev *pdev, unsigned int hwsize) { if (WARN_ON_ONCE(pdev->msix_enabled)) return false; @@ -297,7 +298,7 @@ bool pci_setup_msi_device_domain(struct pci_dev *pdev) if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX)) msi_remove_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN); - return pci_create_device_domain(pdev, &pci_msi_template, 1); + return pci_create_device_domain(pdev, &pci_msi_template, hwsize); } /** diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c index d6ce04054702..6ede55a7c5e6 100644 --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -439,16 +439,16 @@ int __pci_enable_msi_range(struct pci_dev *dev, int minvec, int maxvec, if (nvec < minvec) return -ENOSPC; - if (nvec > maxvec) - nvec = maxvec; - rc = pci_setup_msi_context(dev); if (rc) return rc; - if (!pci_setup_msi_device_domain(dev)) + if (!pci_setup_msi_device_domain(dev, nvec)) return -ENODEV; + if (nvec > maxvec) + nvec = maxvec; + for (;;) { if (affd) { nvec = irq_calc_affinity_vectors(minvec, nvec, affd); diff --git a/drivers/pci/msi/msi.h b/drivers/pci/msi/msi.h index fc70b601e942..0b420b319f50 100644 --- a/drivers/pci/msi/msi.h +++ b/drivers/pci/msi/msi.h @@ -107,7 +107,7 @@ enum support_mode { }; bool pci_msi_domain_supports(struct pci_dev *dev, unsigned int feature_mask, enum support_mode mode); -bool pci_setup_msi_device_domain(struct pci_dev *pdev); +bool pci_setup_msi_device_domain(struct pci_dev *pdev, unsigned int hwsize); bool pci_setup_msix_device_domain(struct pci_dev *pdev, unsigned int hwsize); /* Legacy (!IRQDOMAIN) fallbacks */ diff --git a/drivers/pcmcia/bcm63xx_pcmcia.c b/drivers/pcmcia/bcm63xx_pcmcia.c index 85874b7a9f36..d3baed444646 100644 --- a/drivers/pcmcia/bcm63xx_pcmcia.c +++ b/drivers/pcmcia/bcm63xx_pcmcia.c @@ -268,7 +268,7 @@ static void bcm63xx_pcmcia_poll(struct timer_list *t) struct bcm63xx_pcmcia_socket *skt; unsigned int stat, events; - skt = from_timer(skt, t, timer); + skt = timer_container_of(skt, t, timer); spin_lock_bh(&skt->lock); diff --git a/drivers/pcmcia/electra_cf.c b/drivers/pcmcia/electra_cf.c index 3bdd939dd2f4..2530079d38f4 100644 --- a/drivers/pcmcia/electra_cf.c +++ b/drivers/pcmcia/electra_cf.c @@ -68,7 +68,7 @@ static int electra_cf_ss_init(struct pcmcia_socket *s) /* the timer is primarily to kick this socket's pccardd */ static void electra_cf_timer(struct timer_list *t) { - struct electra_cf_socket *cf = from_timer(cf, t, timer); + struct electra_cf_socket *cf = timer_container_of(cf, t, timer); int present = electra_cf_present(cf); if (present != cf->present) { diff --git a/drivers/pcmcia/omap_cf.c b/drivers/pcmcia/omap_cf.c index f0ccf479f36e..1b1dff56ec7b 100644 --- a/drivers/pcmcia/omap_cf.c +++ b/drivers/pcmcia/omap_cf.c @@ -77,7 +77,7 @@ static int omap_cf_ss_init(struct pcmcia_socket *s) /* the timer is primarily to kick this socket's pccardd */ static void omap_cf_timer(struct timer_list *t) { - struct omap_cf_socket *cf = from_timer(cf, t, timer); + struct omap_cf_socket *cf = timer_container_of(cf, t, timer); unsigned present = omap_cf_present(); if (present != cf->present) { diff --git a/drivers/pcmcia/pd6729.c b/drivers/pcmcia/pd6729.c index a0a2e7f18356..6868b60fd325 100644 --- a/drivers/pcmcia/pd6729.c +++ b/drivers/pcmcia/pd6729.c @@ -236,7 +236,8 @@ static irqreturn_t pd6729_interrupt(int irq, void *dev) static void pd6729_interrupt_wrapper(struct timer_list *t) { - struct pd6729_socket *socket = from_timer(socket, t, poll_timer); + struct pd6729_socket *socket = timer_container_of(socket, t, + poll_timer); pd6729_interrupt(0, (void *)socket); mod_timer(&socket->poll_timer, jiffies + HZ); diff --git a/drivers/pcmcia/soc_common.c b/drivers/pcmcia/soc_common.c index d361124db993..87aa3f667117 100644 --- a/drivers/pcmcia/soc_common.c +++ b/drivers/pcmcia/soc_common.c @@ -460,7 +460,7 @@ static void soc_common_check_status(struct soc_pcmcia_socket *skt) /* Let's poll for events in addition to IRQs since IRQ only is unreliable... */ static void soc_common_pcmcia_poll_event(struct timer_list *t) { - struct soc_pcmcia_socket *skt = from_timer(skt, t, poll_timer); + struct soc_pcmcia_socket *skt = timer_container_of(skt, t, poll_timer); debug(skt, 4, "polling for events\n"); mod_timer(&skt->poll_timer, jiffies + SOC_PCMCIA_POLL_PERIOD); diff --git a/drivers/pcmcia/yenta_socket.c b/drivers/pcmcia/yenta_socket.c index 020ea86c24ec..923ed23570a0 100644 --- a/drivers/pcmcia/yenta_socket.c +++ b/drivers/pcmcia/yenta_socket.c @@ -539,7 +539,8 @@ static irqreturn_t yenta_interrupt(int irq, void *dev_id) static void yenta_interrupt_wrapper(struct timer_list *t) { - struct yenta_socket *socket = from_timer(socket, t, poll_timer); + struct yenta_socket *socket = timer_container_of(socket, t, + poll_timer); yenta_interrupt(0, (void *)socket); socket->poll_timer.expires = jiffies + HZ; diff --git a/drivers/pinctrl/pinctrl-st.c b/drivers/pinctrl/pinctrl-st.c index fe2d52e434db..8a2ef74862d3 100644 --- a/drivers/pinctrl/pinctrl-st.c +++ b/drivers/pinctrl/pinctrl-st.c @@ -374,11 +374,6 @@ static struct st_pio_control *st_get_pio_control( } /* Low level functions.. */ -static inline int st_gpio_bank(int gpio) -{ - return gpio/ST_GPIO_PINS_PER_BANK; -} - static inline int st_gpio_pin(int gpio) { return gpio%ST_GPIO_PINS_PER_BANK; diff --git a/drivers/pinctrl/pinctrl-tb10x.c b/drivers/pinctrl/pinctrl-tb10x.c index d6bb8f58978d..4edb20e61951 100644 --- a/drivers/pinctrl/pinctrl-tb10x.c +++ b/drivers/pinctrl/pinctrl-tb10x.c @@ -823,7 +823,7 @@ static struct platform_driver tb10x_pinctrl_pdrv = { .remove = tb10x_pinctrl_remove, .driver = { .name = "tb10x_pinctrl", - .of_match_table = of_match_ptr(tb10x_pinctrl_dt_ids), + .of_match_table = tb10x_pinctrl_dt_ids, } }; diff --git a/drivers/pinctrl/qcom/pinctrl-apq8064.c b/drivers/pinctrl/qcom/pinctrl-apq8064.c index 20c3b9025044..3654913f1ae5 100644 --- a/drivers/pinctrl/qcom/pinctrl-apq8064.c +++ b/drivers/pinctrl/qcom/pinctrl-apq8064.c @@ -629,7 +629,6 @@ static struct platform_driver apq8064_pinctrl_driver = { .of_match_table = apq8064_pinctrl_of_match, }, .probe = apq8064_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init apq8064_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-apq8084.c b/drivers/pinctrl/qcom/pinctrl-apq8084.c index 3fc0a40762b6..27693cd64881 100644 --- a/drivers/pinctrl/qcom/pinctrl-apq8084.c +++ b/drivers/pinctrl/qcom/pinctrl-apq8084.c @@ -1207,7 +1207,6 @@ static struct platform_driver apq8084_pinctrl_driver = { .of_match_table = apq8084_pinctrl_of_match, }, .probe = apq8084_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init apq8084_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq4019.c b/drivers/pinctrl/qcom/pinctrl-ipq4019.c index 1f7944dd829d..6ede3149b6e1 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq4019.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq4019.c @@ -710,7 +710,6 @@ static struct platform_driver ipq4019_pinctrl_driver = { .of_match_table = ipq4019_pinctrl_of_match, }, .probe = ipq4019_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq4019_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq5018.c b/drivers/pinctrl/qcom/pinctrl-ipq5018.c index e2951f81c3ee..10b99d5d8a11 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq5018.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq5018.c @@ -754,7 +754,6 @@ static struct platform_driver ipq5018_pinctrl_driver = { .of_match_table = ipq5018_pinctrl_of_match, }, .probe = ipq5018_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq5018_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq5332.c b/drivers/pinctrl/qcom/pinctrl-ipq5332.c index 625f8014051f..1ac2fc09c119 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq5332.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq5332.c @@ -834,7 +834,6 @@ static struct platform_driver ipq5332_pinctrl_driver = { .of_match_table = ipq5332_pinctrl_of_match, }, .probe = ipq5332_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq5332_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq5424.c b/drivers/pinctrl/qcom/pinctrl-ipq5424.c index 0d610b076da3..7ff1f8acc1a3 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq5424.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq5424.c @@ -791,7 +791,6 @@ static struct platform_driver ipq5424_pinctrl_driver = { .of_match_table = ipq5424_pinctrl_of_match, }, .probe = ipq5424_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq5424_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq6018.c b/drivers/pinctrl/qcom/pinctrl-ipq6018.c index 0ad08647dbcd..a4ba980252e1 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq6018.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq6018.c @@ -1080,7 +1080,6 @@ static struct platform_driver ipq6018_pinctrl_driver = { .of_match_table = ipq6018_pinctrl_of_match, }, .probe = ipq6018_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq6018_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq8064.c b/drivers/pinctrl/qcom/pinctrl-ipq8064.c index e2bb94e86aef..0a9e357e64c6 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq8064.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq8064.c @@ -631,7 +631,6 @@ static struct platform_driver ipq8064_pinctrl_driver = { .of_match_table = ipq8064_pinctrl_of_match, }, .probe = ipq8064_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq8064_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq8074.c b/drivers/pinctrl/qcom/pinctrl-ipq8074.c index 337f3a1c92c1..482f13282fc2 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq8074.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq8074.c @@ -1041,7 +1041,6 @@ static struct platform_driver ipq8074_pinctrl_driver = { .of_match_table = ipq8074_pinctrl_of_match, }, .probe = ipq8074_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq8074_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-ipq9574.c b/drivers/pinctrl/qcom/pinctrl-ipq9574.c index e2491617b236..89c05d8eb550 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq9574.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq9574.c @@ -799,7 +799,6 @@ static struct platform_driver ipq9574_pinctrl_driver = { .of_match_table = ipq9574_pinctrl_of_match, }, .probe = ipq9574_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init ipq9574_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-mdm9607.c b/drivers/pinctrl/qcom/pinctrl-mdm9607.c index e7cd3ef1cf3e..3e18ba124fed 100644 --- a/drivers/pinctrl/qcom/pinctrl-mdm9607.c +++ b/drivers/pinctrl/qcom/pinctrl-mdm9607.c @@ -1059,7 +1059,6 @@ static struct platform_driver mdm9607_pinctrl_driver = { .of_match_table = mdm9607_pinctrl_of_match, }, .probe = mdm9607_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init mdm9607_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-mdm9615.c b/drivers/pinctrl/qcom/pinctrl-mdm9615.c index 0a2ae383d3d5..bea1ca3d1b7f 100644 --- a/drivers/pinctrl/qcom/pinctrl-mdm9615.c +++ b/drivers/pinctrl/qcom/pinctrl-mdm9615.c @@ -446,7 +446,6 @@ static struct platform_driver mdm9615_pinctrl_driver = { .of_match_table = mdm9615_pinctrl_of_match, }, .probe = mdm9615_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init mdm9615_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm.c b/drivers/pinctrl/qcom/pinctrl-msm.c index f012ea88aa22..5c4687de1464 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.c +++ b/drivers/pinctrl/qcom/pinctrl-msm.c @@ -1442,7 +1442,7 @@ static int msm_gpio_init(struct msm_pinctrl *pctrl) girq->handler = handle_bad_irq; girq->parents[0] = pctrl->irq; - ret = gpiochip_add_data(&pctrl->chip, pctrl); + ret = devm_gpiochip_add_data(pctrl->dev, &pctrl->chip, pctrl); if (ret) { dev_err(pctrl->dev, "Failed register gpiochip\n"); return ret; @@ -1463,7 +1463,6 @@ static int msm_gpio_init(struct msm_pinctrl *pctrl) dev_name(pctrl->dev), 0, 0, chip->ngpio); if (ret) { dev_err(pctrl->dev, "Failed to add pin range\n"); - gpiochip_remove(&pctrl->chip); return ret; } } @@ -1599,13 +1598,5 @@ int msm_pinctrl_probe(struct platform_device *pdev, } EXPORT_SYMBOL(msm_pinctrl_probe); -void msm_pinctrl_remove(struct platform_device *pdev) -{ - struct msm_pinctrl *pctrl = platform_get_drvdata(pdev); - - gpiochip_remove(&pctrl->chip); -} -EXPORT_SYMBOL(msm_pinctrl_remove); - MODULE_DESCRIPTION("Qualcomm Technologies, Inc. TLMM driver"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/pinctrl/qcom/pinctrl-msm.h b/drivers/pinctrl/qcom/pinctrl-msm.h index 63852ed70295..d7dc0947bb16 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.h +++ b/drivers/pinctrl/qcom/pinctrl-msm.h @@ -171,6 +171,5 @@ extern const struct dev_pm_ops msm_pinctrl_dev_pm_ops; int msm_pinctrl_probe(struct platform_device *pdev, const struct msm_pinctrl_soc_data *soc_data); -void msm_pinctrl_remove(struct platform_device *pdev); #endif diff --git a/drivers/pinctrl/qcom/pinctrl-msm8226.c b/drivers/pinctrl/qcom/pinctrl-msm8226.c index 64fee70f1772..f9a957347340 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8226.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8226.c @@ -654,7 +654,6 @@ static struct platform_driver msm8226_pinctrl_driver = { .of_match_table = msm8226_pinctrl_of_match, }, .probe = msm8226_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8226_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8660.c b/drivers/pinctrl/qcom/pinctrl-msm8660.c index 999a5f867eb5..4dbc19ffd80e 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8660.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8660.c @@ -981,7 +981,6 @@ static struct platform_driver msm8660_pinctrl_driver = { .of_match_table = msm8660_pinctrl_of_match, }, .probe = msm8660_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8660_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8909.c b/drivers/pinctrl/qcom/pinctrl-msm8909.c index 756856d20d6b..0aa4f77b774f 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8909.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8909.c @@ -929,7 +929,6 @@ static struct platform_driver msm8909_pinctrl_driver = { .of_match_table = msm8909_pinctrl_of_match, }, .probe = msm8909_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8909_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8916.c b/drivers/pinctrl/qcom/pinctrl-msm8916.c index cea5c54f92fe..0dfc6dd33d58 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8916.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8916.c @@ -969,7 +969,6 @@ static struct platform_driver msm8916_pinctrl_driver = { .of_match_table = msm8916_pinctrl_of_match, }, .probe = msm8916_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8916_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8917.c b/drivers/pinctrl/qcom/pinctrl-msm8917.c index 350636807b07..2e1a94ab18b2 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8917.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8917.c @@ -1607,7 +1607,6 @@ static struct platform_driver msm8917_pinctrl_driver = { .of_match_table = msm8917_pinctrl_of_match, }, .probe = msm8917_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8917_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8953.c b/drivers/pinctrl/qcom/pinctrl-msm8953.c index 998351bdfee1..956383341a7a 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8953.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8953.c @@ -1816,7 +1816,6 @@ static struct platform_driver msm8953_pinctrl_driver = { .of_match_table = msm8953_pinctrl_of_match, }, .probe = msm8953_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8953_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8960.c b/drivers/pinctrl/qcom/pinctrl-msm8960.c index ebe230b3b437..a937ea867de7 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8960.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8960.c @@ -1246,7 +1246,6 @@ static struct platform_driver msm8960_pinctrl_driver = { .of_match_table = msm8960_pinctrl_of_match, }, .probe = msm8960_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8960_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8976.c b/drivers/pinctrl/qcom/pinctrl-msm8976.c index c30d80e4e98c..3bcb03387781 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8976.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8976.c @@ -1096,7 +1096,6 @@ static struct platform_driver msm8976_pinctrl_driver = { .of_match_table = msm8976_pinctrl_of_match, }, .probe = msm8976_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8976_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8994.c b/drivers/pinctrl/qcom/pinctrl-msm8994.c index b1a6759ab4a5..7a3b6cbccb68 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8994.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8994.c @@ -1343,7 +1343,6 @@ static struct platform_driver msm8994_pinctrl_driver = { .of_match_table = msm8994_pinctrl_of_match, }, .probe = msm8994_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8994_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8996.c b/drivers/pinctrl/qcom/pinctrl-msm8996.c index 1b5d80eaab83..d86d83106d3b 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8996.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8996.c @@ -1920,7 +1920,6 @@ static struct platform_driver msm8996_pinctrl_driver = { .of_match_table = msm8996_pinctrl_of_match, }, .probe = msm8996_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8996_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8998.c b/drivers/pinctrl/qcom/pinctrl-msm8998.c index b7cbf32b3125..1daee815888f 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8998.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8998.c @@ -1535,7 +1535,6 @@ static struct platform_driver msm8998_pinctrl_driver = { .of_match_table = msm8998_pinctrl_of_match, }, .probe = msm8998_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8998_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-msm8x74.c b/drivers/pinctrl/qcom/pinctrl-msm8x74.c index 238c83f6ec4f..8253aa25775b 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm8x74.c +++ b/drivers/pinctrl/qcom/pinctrl-msm8x74.c @@ -1083,7 +1083,6 @@ static struct platform_driver msm8x74_pinctrl_driver = { .of_match_table = msm8x74_pinctrl_of_match, }, .probe = msm8x74_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init msm8x74_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qcm2290.c b/drivers/pinctrl/qcom/pinctrl-qcm2290.c index f885af571ec9..eeeec6434f6a 100644 --- a/drivers/pinctrl/qcom/pinctrl-qcm2290.c +++ b/drivers/pinctrl/qcom/pinctrl-qcm2290.c @@ -167,6 +167,10 @@ static const struct pinctrl_pin_desc qcm2290_pins[] = { PINCTRL_PIN(62, "GPIO_62"), PINCTRL_PIN(63, "GPIO_63"), PINCTRL_PIN(64, "GPIO_64"), + PINCTRL_PIN(65, "GPIO_65"), + PINCTRL_PIN(66, "GPIO_66"), + PINCTRL_PIN(67, "GPIO_67"), + PINCTRL_PIN(68, "GPIO_68"), PINCTRL_PIN(69, "GPIO_69"), PINCTRL_PIN(70, "GPIO_70"), PINCTRL_PIN(71, "GPIO_71"), @@ -181,12 +185,17 @@ static const struct pinctrl_pin_desc qcm2290_pins[] = { PINCTRL_PIN(80, "GPIO_80"), PINCTRL_PIN(81, "GPIO_81"), PINCTRL_PIN(82, "GPIO_82"), + PINCTRL_PIN(83, "GPIO_83"), + PINCTRL_PIN(84, "GPIO_84"), + PINCTRL_PIN(85, "GPIO_85"), PINCTRL_PIN(86, "GPIO_86"), PINCTRL_PIN(87, "GPIO_87"), PINCTRL_PIN(88, "GPIO_88"), PINCTRL_PIN(89, "GPIO_89"), PINCTRL_PIN(90, "GPIO_90"), PINCTRL_PIN(91, "GPIO_91"), + PINCTRL_PIN(92, "GPIO_92"), + PINCTRL_PIN(93, "GPIO_93"), PINCTRL_PIN(94, "GPIO_94"), PINCTRL_PIN(95, "GPIO_95"), PINCTRL_PIN(96, "GPIO_96"), @@ -1125,7 +1134,6 @@ static struct platform_driver qcm2290_pinctrl_driver = { .of_match_table = qcm2290_pinctrl_of_match, }, .probe = qcm2290_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init qcm2290_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qcs404.c b/drivers/pinctrl/qcom/pinctrl-qcs404.c index ae7224012f8a..54e3b4435349 100644 --- a/drivers/pinctrl/qcom/pinctrl-qcs404.c +++ b/drivers/pinctrl/qcom/pinctrl-qcs404.c @@ -1644,7 +1644,6 @@ static struct platform_driver qcs404_pinctrl_driver = { .of_match_table = qcs404_pinctrl_of_match, }, .probe = qcs404_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init qcs404_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qcs615.c b/drivers/pinctrl/qcom/pinctrl-qcs615.c index 17ca743c2210..2a943bc46a62 100644 --- a/drivers/pinctrl/qcom/pinctrl-qcs615.c +++ b/drivers/pinctrl/qcom/pinctrl-qcs615.c @@ -1087,7 +1087,6 @@ static struct platform_driver qcs615_tlmm_driver = { .of_match_table = qcs615_tlmm_of_match, }, .probe = qcs615_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init qcs615_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qcs8300.c b/drivers/pinctrl/qcom/pinctrl-qcs8300.c index 5f5f7c4ac644..d6437e26392b 100644 --- a/drivers/pinctrl/qcom/pinctrl-qcs8300.c +++ b/drivers/pinctrl/qcom/pinctrl-qcs8300.c @@ -1227,7 +1227,6 @@ static struct platform_driver qcs8300_pinctrl_driver = { .of_match_table = qcs8300_pinctrl_of_match, }, .probe = qcs8300_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init qcs8300_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c b/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c index b5808fcfb13c..9ecc4d40e4dc 100644 --- a/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c +++ b/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c @@ -145,7 +145,6 @@ static struct platform_driver qdf2xxx_pinctrl_driver = { .acpi_match_table = qdf2xxx_acpi_ids, }, .probe = qdf2xxx_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init qdf2xxx_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-qdu1000.c b/drivers/pinctrl/qcom/pinctrl-qdu1000.c index 47bc529ef550..eacb89fa3888 100644 --- a/drivers/pinctrl/qcom/pinctrl-qdu1000.c +++ b/drivers/pinctrl/qcom/pinctrl-qdu1000.c @@ -1248,7 +1248,6 @@ static struct platform_driver qdu1000_tlmm_driver = { .of_match_table = qdu1000_tlmm_of_match, }, .probe = qdu1000_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init qdu1000_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sa8775p.c b/drivers/pinctrl/qcom/pinctrl-sa8775p.c index a5b38221aea8..1b62eb3e6620 100644 --- a/drivers/pinctrl/qcom/pinctrl-sa8775p.c +++ b/drivers/pinctrl/qcom/pinctrl-sa8775p.c @@ -1540,7 +1540,6 @@ static struct platform_driver sa8775p_pinctrl_driver = { .of_match_table = sa8775p_pinctrl_of_match, }, .probe = sa8775p_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sa8775p_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sar2130p.c b/drivers/pinctrl/qcom/pinctrl-sar2130p.c index 19a2e37826c7..3dd1b5e5cfee 100644 --- a/drivers/pinctrl/qcom/pinctrl-sar2130p.c +++ b/drivers/pinctrl/qcom/pinctrl-sar2130p.c @@ -1486,7 +1486,6 @@ static struct platform_driver sar2130p_tlmm_driver = { .of_match_table = sar2130p_tlmm_of_match, }, .probe = sar2130p_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sar2130p_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sc7180.c b/drivers/pinctrl/qcom/pinctrl-sc7180.c index 6eb0c73791c0..c43fe10b71ad 100644 --- a/drivers/pinctrl/qcom/pinctrl-sc7180.c +++ b/drivers/pinctrl/qcom/pinctrl-sc7180.c @@ -1159,7 +1159,6 @@ static struct platform_driver sc7180_pinctrl_driver = { .of_match_table = sc7180_pinctrl_of_match, }, .probe = sc7180_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sc7180_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sc7280.c b/drivers/pinctrl/qcom/pinctrl-sc7280.c index 0c10eeb60b55..1b070e9d41f5 100644 --- a/drivers/pinctrl/qcom/pinctrl-sc7280.c +++ b/drivers/pinctrl/qcom/pinctrl-sc7280.c @@ -1505,7 +1505,6 @@ static struct platform_driver sc7280_pinctrl_driver = { .of_match_table = sc7280_pinctrl_of_match, }, .probe = sc7280_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sc7280_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sc8180x.c b/drivers/pinctrl/qcom/pinctrl-sc8180x.c index d6a79ad41a40..26dd165d1543 100644 --- a/drivers/pinctrl/qcom/pinctrl-sc8180x.c +++ b/drivers/pinctrl/qcom/pinctrl-sc8180x.c @@ -1720,7 +1720,6 @@ static struct platform_driver sc8180x_pinctrl_driver = { .acpi_match_table = sc8180x_pinctrl_acpi_match, }, .probe = sc8180x_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sc8180x_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sc8280xp.c b/drivers/pinctrl/qcom/pinctrl-sc8280xp.c index 96f4fb5a5d29..6ccd7e5648d4 100644 --- a/drivers/pinctrl/qcom/pinctrl-sc8280xp.c +++ b/drivers/pinctrl/qcom/pinctrl-sc8280xp.c @@ -1926,7 +1926,6 @@ static struct platform_driver sc8280xp_pinctrl_driver = { .of_match_table = sc8280xp_pinctrl_of_match, }, .probe = sc8280xp_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sc8280xp_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdm660.c b/drivers/pinctrl/qcom/pinctrl-sdm660.c index 907e4ffca5e7..1a78288f1bc8 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdm660.c +++ b/drivers/pinctrl/qcom/pinctrl-sdm660.c @@ -1442,7 +1442,6 @@ static struct platform_driver sdm660_pinctrl_driver = { .of_match_table = sdm660_pinctrl_of_match, }, .probe = sdm660_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdm660_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdm670.c b/drivers/pinctrl/qcom/pinctrl-sdm670.c index c76183ba95e1..0fe1fa94cd6d 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdm670.c +++ b/drivers/pinctrl/qcom/pinctrl-sdm670.c @@ -1337,7 +1337,6 @@ static struct platform_driver sdm670_pinctrl_driver = { .of_match_table = sdm670_pinctrl_of_match, }, .probe = sdm670_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdm670_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdm845.c b/drivers/pinctrl/qcom/pinctrl-sdm845.c index cc05c415ed15..0446e291aa48 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdm845.c +++ b/drivers/pinctrl/qcom/pinctrl-sdm845.c @@ -1351,7 +1351,6 @@ static struct platform_driver sdm845_pinctrl_driver = { .acpi_match_table = ACPI_PTR(sdm845_pinctrl_acpi_match), }, .probe = sdm845_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdm845_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdx55.c b/drivers/pinctrl/qcom/pinctrl-sdx55.c index 8826db9d21d0..2c17bf889146 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdx55.c +++ b/drivers/pinctrl/qcom/pinctrl-sdx55.c @@ -990,7 +990,6 @@ static struct platform_driver sdx55_pinctrl_driver = { .of_match_table = sdx55_pinctrl_of_match, }, .probe = sdx55_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdx55_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdx65.c b/drivers/pinctrl/qcom/pinctrl-sdx65.c index f6f319c997fc..85b5c0206dbd 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdx65.c +++ b/drivers/pinctrl/qcom/pinctrl-sdx65.c @@ -939,7 +939,6 @@ static struct platform_driver sdx65_pinctrl_driver = { .of_match_table = sdx65_pinctrl_of_match, }, .probe = sdx65_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdx65_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sdx75.c b/drivers/pinctrl/qcom/pinctrl-sdx75.c index 3cfe8c7f04df..ab13a3a57a83 100644 --- a/drivers/pinctrl/qcom/pinctrl-sdx75.c +++ b/drivers/pinctrl/qcom/pinctrl-sdx75.c @@ -1124,7 +1124,6 @@ static struct platform_driver sdx75_pinctrl_driver = { .of_match_table = sdx75_pinctrl_of_match, }, .probe = sdx75_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sdx75_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm4450.c b/drivers/pinctrl/qcom/pinctrl-sm4450.c index 622f20e6f6f8..1ecdf1ab4f27 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm4450.c +++ b/drivers/pinctrl/qcom/pinctrl-sm4450.c @@ -994,7 +994,6 @@ static struct platform_driver sm4450_tlmm_driver = { .of_match_table = sm4450_tlmm_of_match, }, .probe = sm4450_tlmm_probe, - .remove = msm_pinctrl_remove, }; MODULE_DEVICE_TABLE(of, sm4450_tlmm_of_match); diff --git a/drivers/pinctrl/qcom/pinctrl-sm6115.c b/drivers/pinctrl/qcom/pinctrl-sm6115.c index 4e91c75ad952..c273efa43996 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm6115.c +++ b/drivers/pinctrl/qcom/pinctrl-sm6115.c @@ -907,7 +907,6 @@ static struct platform_driver sm6115_tlmm_driver = { .of_match_table = sm6115_tlmm_of_match, }, .probe = sm6115_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm6115_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm6125.c b/drivers/pinctrl/qcom/pinctrl-sm6125.c index c188842047aa..5092f20e0c1b 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm6125.c +++ b/drivers/pinctrl/qcom/pinctrl-sm6125.c @@ -1266,7 +1266,6 @@ static struct platform_driver sm6125_tlmm_driver = { .of_match_table = sm6125_tlmm_of_match, }, .probe = sm6125_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm6125_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm6350.c b/drivers/pinctrl/qcom/pinctrl-sm6350.c index f3828c07b134..ba4686c86c54 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm6350.c +++ b/drivers/pinctrl/qcom/pinctrl-sm6350.c @@ -1373,7 +1373,6 @@ static struct platform_driver sm6350_tlmm_driver = { .of_match_table = sm6350_tlmm_of_match, }, .probe = sm6350_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm6350_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm6375.c b/drivers/pinctrl/qcom/pinctrl-sm6375.c index c82c8516932e..49031571e65e 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm6375.c +++ b/drivers/pinctrl/qcom/pinctrl-sm6375.c @@ -1516,7 +1516,6 @@ static struct platform_driver sm6375_tlmm_driver = { .of_match_table = sm6375_tlmm_of_match, }, .probe = sm6375_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm6375_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm7150.c b/drivers/pinctrl/qcom/pinctrl-sm7150.c index 3c7fd8af6635..6e89966cd70e 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm7150.c +++ b/drivers/pinctrl/qcom/pinctrl-sm7150.c @@ -1255,7 +1255,6 @@ static struct platform_driver sm7150_tlmm_driver = { .of_match_table = sm7150_tlmm_of_match, }, .probe = sm7150_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm7150_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8150.c b/drivers/pinctrl/qcom/pinctrl-sm8150.c index 01aea9c70b7a..794ed99463f7 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8150.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8150.c @@ -1542,7 +1542,6 @@ static struct platform_driver sm8150_pinctrl_driver = { .of_match_table = sm8150_pinctrl_of_match, }, .probe = sm8150_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8150_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8250.c b/drivers/pinctrl/qcom/pinctrl-sm8250.c index e9961a49ff98..fb6f005d64f5 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8250.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8250.c @@ -1351,7 +1351,6 @@ static struct platform_driver sm8250_pinctrl_driver = { .of_match_table = sm8250_pinctrl_of_match, }, .probe = sm8250_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8250_pinctrl_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8350.c b/drivers/pinctrl/qcom/pinctrl-sm8350.c index 9c69458bd910..c8a3f39ce6f1 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8350.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8350.c @@ -1642,7 +1642,6 @@ static struct platform_driver sm8350_tlmm_driver = { .of_match_table = sm8350_tlmm_of_match, }, .probe = sm8350_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8350_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8450.c b/drivers/pinctrl/qcom/pinctrl-sm8450.c index d11bb1ee9e3d..f2e52d5a0f93 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8450.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8450.c @@ -1677,7 +1677,6 @@ static struct platform_driver sm8450_tlmm_driver = { .of_match_table = sm8450_tlmm_of_match, }, .probe = sm8450_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8450_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8550.c b/drivers/pinctrl/qcom/pinctrl-sm8550.c index 3c847d9cb5d9..1b4496cb39eb 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8550.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8550.c @@ -1762,7 +1762,6 @@ static struct platform_driver sm8550_tlmm_driver = { .of_match_table = sm8550_tlmm_of_match, }, .probe = sm8550_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8550_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8650.c b/drivers/pinctrl/qcom/pinctrl-sm8650.c index 104708252d12..449a0077f4b1 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8650.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8650.c @@ -1742,7 +1742,6 @@ static struct platform_driver sm8650_tlmm_driver = { .of_match_table = sm8650_tlmm_of_match, }, .probe = sm8650_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8650_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-sm8750.c b/drivers/pinctrl/qcom/pinctrl-sm8750.c index b94fb4ee0ec3..8516693d1db5 100644 --- a/drivers/pinctrl/qcom/pinctrl-sm8750.c +++ b/drivers/pinctrl/qcom/pinctrl-sm8750.c @@ -1711,7 +1711,6 @@ static struct platform_driver sm8750_tlmm_driver = { .of_match_table = sm8750_tlmm_of_match, }, .probe = sm8750_tlmm_probe, - .remove = msm_pinctrl_remove, }; static int __init sm8750_tlmm_init(void) diff --git a/drivers/pinctrl/qcom/pinctrl-x1e80100.c b/drivers/pinctrl/qcom/pinctrl-x1e80100.c index 419cb8facb2f..d4b215f34c39 100644 --- a/drivers/pinctrl/qcom/pinctrl-x1e80100.c +++ b/drivers/pinctrl/qcom/pinctrl-x1e80100.c @@ -1861,7 +1861,6 @@ static struct platform_driver x1e80100_pinctrl_driver = { .of_match_table = x1e80100_pinctrl_of_match, }, .probe = x1e80100_pinctrl_probe, - .remove = msm_pinctrl_remove, }; static int __init x1e80100_pinctrl_init(void) diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi-dt.c b/drivers/pinctrl/sunxi/pinctrl-sunxi-dt.c index 1833078f6877..4e34b0cd3b73 100644 --- a/drivers/pinctrl/sunxi/pinctrl-sunxi-dt.c +++ b/drivers/pinctrl/sunxi/pinctrl-sunxi-dt.c @@ -143,7 +143,7 @@ static struct sunxi_desc_pin *init_pins_table(struct device *dev, */ static int prepare_function_table(struct device *dev, struct device_node *pnode, struct sunxi_desc_pin *pins, int npins, - const u8 *irq_bank_muxes) + unsigned pin_base, const u8 *irq_bank_muxes) { struct device_node *node; struct property *prop; @@ -166,7 +166,7 @@ static int prepare_function_table(struct device *dev, struct device_node *pnode, */ for (i = 0; i < npins; i++) { struct sunxi_desc_pin *pin = &pins[i]; - int bank = pin->pin.number / PINS_PER_BANK; + int bank = (pin->pin.number - pin_base) / PINS_PER_BANK; if (irq_bank_muxes[bank]) { pin->variant++; @@ -211,7 +211,7 @@ static int prepare_function_table(struct device *dev, struct device_node *pnode, last_bank = 0; for (i = 0; i < npins; i++) { struct sunxi_desc_pin *pin = &pins[i]; - int bank = pin->pin.number / PINS_PER_BANK; + int bank = (pin->pin.number - pin_base) / PINS_PER_BANK; int lastfunc = pin->variant + 1; int irq_mux = irq_bank_muxes[bank]; @@ -353,7 +353,7 @@ int sunxi_pinctrl_dt_table_init(struct platform_device *pdev, return PTR_ERR(pins); ret = prepare_function_table(&pdev->dev, pnode, pins, desc->npins, - irq_bank_muxes); + desc->pin_base, irq_bank_muxes); if (ret) return ret; diff --git a/drivers/platform/loongarch/loongson-laptop.c b/drivers/platform/loongarch/loongson-laptop.c index 99203584949d..61b18ac206c9 100644 --- a/drivers/platform/loongarch/loongson-laptop.c +++ b/drivers/platform/loongarch/loongson-laptop.c @@ -56,8 +56,7 @@ static struct input_dev *generic_inputdev; static acpi_handle hotkey_handle; static struct key_entry hotkey_keycode_map[GENERIC_HOTKEY_MAP_MAX]; -int loongson_laptop_turn_on_backlight(void); -int loongson_laptop_turn_off_backlight(void); +static bool bl_powered; static int loongson_laptop_backlight_update(struct backlight_device *bd); /* 2. ACPI Helpers and device model */ @@ -354,16 +353,42 @@ static int ec_backlight_level(u8 level) return level; } +static int ec_backlight_set_power(bool state) +{ + int status; + union acpi_object arg0 = { ACPI_TYPE_INTEGER }; + struct acpi_object_list args = { 1, &arg0 }; + + arg0.integer.value = state; + status = acpi_evaluate_object(NULL, "\\BLSW", &args, NULL); + if (ACPI_FAILURE(status)) { + pr_info("Loongson lvds error: 0x%x\n", status); + return -EIO; + } + + return 0; +} + static int loongson_laptop_backlight_update(struct backlight_device *bd) { - int lvl = ec_backlight_level(bd->props.brightness); + bool target_powered = !backlight_is_blank(bd); + int ret = 0, lvl = ec_backlight_level(bd->props.brightness); if (lvl < 0) return -EIO; + if (ec_set_brightness(lvl)) return -EIO; - return 0; + if (target_powered != bl_powered) { + ret = ec_backlight_set_power(target_powered); + if (ret < 0) + return ret; + + bl_powered = target_powered; + } + + return ret; } static int loongson_laptop_get_brightness(struct backlight_device *bd) @@ -384,7 +409,7 @@ static const struct backlight_ops backlight_laptop_ops = { static int laptop_backlight_register(void) { - int status = 0; + int status = 0, ret; struct backlight_properties props; memset(&props, 0, sizeof(props)); @@ -392,44 +417,20 @@ static int laptop_backlight_register(void) if (!acpi_evalf(hotkey_handle, &status, "ECLL", "d")) return -EIO; - props.brightness = 1; + ret = ec_backlight_set_power(true); + if (ret) + return ret; + + bl_powered = true; + props.max_brightness = status; + props.brightness = ec_get_brightness(); + props.power = BACKLIGHT_POWER_ON; props.type = BACKLIGHT_PLATFORM; backlight_device_register("loongson_laptop", NULL, NULL, &backlight_laptop_ops, &props); - return 0; -} - -int loongson_laptop_turn_on_backlight(void) -{ - int status; - union acpi_object arg0 = { ACPI_TYPE_INTEGER }; - struct acpi_object_list args = { 1, &arg0 }; - - arg0.integer.value = 1; - status = acpi_evaluate_object(NULL, "\\BLSW", &args, NULL); - if (ACPI_FAILURE(status)) { - pr_info("Loongson lvds error: 0x%x\n", status); - return -ENODEV; - } - - return 0; -} - -int loongson_laptop_turn_off_backlight(void) -{ - int status; - union acpi_object arg0 = { ACPI_TYPE_INTEGER }; - struct acpi_object_list args = { 1, &arg0 }; - - arg0.integer.value = 0; - status = acpi_evaluate_object(NULL, "\\BLSW", &args, NULL); - if (ACPI_FAILURE(status)) { - pr_info("Loongson lvds error: 0x%x\n", status); - return -ENODEV; - } return 0; } @@ -611,11 +612,17 @@ static int __init generic_acpi_laptop_init(void) static void __exit generic_acpi_laptop_exit(void) { + int i; + if (generic_inputdev) { - if (input_device_registered) - input_unregister_device(generic_inputdev); - else + if (!input_device_registered) { input_free_device(generic_inputdev); + } else { + input_unregister_device(generic_inputdev); + + for (i = 0; i < ARRAY_SIZE(generic_sub_drivers); i++) + generic_subdriver_exit(&generic_sub_drivers[i]); + } } } diff --git a/drivers/platform/x86/intel_ips.c b/drivers/platform/x86/intel_ips.c index 9506f28fb7d8..b1b2d9caba7b 100644 --- a/drivers/platform/x86/intel_ips.c +++ b/drivers/platform/x86/intel_ips.c @@ -934,7 +934,7 @@ static u32 calc_avg_power(struct ips_driver *ips, u32 *array) static void monitor_timeout(struct timer_list *t) { - struct ips_driver *ips = from_timer(ips, t, timer); + struct ips_driver *ips = timer_container_of(ips, t, timer); wake_up_process(ips->monitor); } diff --git a/drivers/pps/clients/pps-gpio.c b/drivers/pps/clients/pps-gpio.c index 374ceefd6f2a..47d9891de368 100644 --- a/drivers/pps/clients/pps-gpio.c +++ b/drivers/pps/clients/pps-gpio.c @@ -98,7 +98,7 @@ static void pps_gpio_echo_timer_callback(struct timer_list *t) { const struct pps_gpio_device_data *info; - info = from_timer(info, t, echo_timer); + info = timer_container_of(info, t, echo_timer); gpiod_set_value(info->echo_pin, 0); } diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index ce804438c32d..1e7f72e57557 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -1526,7 +1526,7 @@ ptp_ocp_utc_distribute(struct ptp_ocp *bp, u32 val) static void ptp_ocp_watchdog(struct timer_list *t) { - struct ptp_ocp *bp = from_timer(bp, t, watchdog); + struct ptp_ocp *bp = timer_container_of(bp, t, watchdog); unsigned long flags; u32 status, utc_offset; diff --git a/drivers/ptp/ptp_private.h b/drivers/ptp/ptp_private.h index 18934e28469e..528d86a33f37 100644 --- a/drivers/ptp/ptp_private.h +++ b/drivers/ptp/ptp_private.h @@ -98,17 +98,7 @@ static inline int queue_cnt(const struct timestamp_event_queue *q) /* Check if ptp virtual clock is in use */ static inline bool ptp_vclock_in_use(struct ptp_clock *ptp) { - bool in_use = false; - - if (mutex_lock_interruptible(&ptp->n_vclocks_mux)) - return true; - - if (!ptp->is_virtual_clock && ptp->n_vclocks) - in_use = true; - - mutex_unlock(&ptp->n_vclocks_mux); - - return in_use; + return !ptp->is_virtual_clock; } /* Check if ptp clock shall be free running */ diff --git a/drivers/regulator/max20086-regulator.c b/drivers/regulator/max20086-regulator.c index b4fe76e33ff2..fcdd2d0317a5 100644 --- a/drivers/regulator/max20086-regulator.c +++ b/drivers/regulator/max20086-regulator.c @@ -5,6 +5,7 @@ // Copyright (C) 2022 Laurent Pinchart <laurent.pinchart@idesonboard.com> // Copyright (C) 2018 Avnet, Inc. +#include <linux/cleanup.h> #include <linux/err.h> #include <linux/gpio/consumer.h> #include <linux/i2c.h> @@ -133,11 +134,11 @@ static int max20086_regulators_register(struct max20086 *chip) static int max20086_parse_regulators_dt(struct max20086 *chip, bool *boot_on) { struct of_regulator_match *matches; - struct device_node *node; unsigned int i; int ret; - node = of_get_child_by_name(chip->dev->of_node, "regulators"); + struct device_node *node __free(device_node) = + of_get_child_by_name(chip->dev->of_node, "regulators"); if (!node) { dev_err(chip->dev, "regulators node not found\n"); return -ENODEV; @@ -153,7 +154,6 @@ static int max20086_parse_regulators_dt(struct max20086 *chip, bool *boot_on) ret = of_regulator_match(chip->dev, node, matches, chip->info->num_outputs); - of_node_put(node); if (ret < 0) { dev_err(chip->dev, "Failed to match regulators\n"); return -EINVAL; diff --git a/drivers/rtc/dev.c b/drivers/rtc/dev.c index 0eeae5bcc3aa..baf1a8ca8b2b 100644 --- a/drivers/rtc/dev.c +++ b/drivers/rtc/dev.c @@ -72,7 +72,7 @@ static void rtc_uie_task(struct work_struct *work) static void rtc_uie_timer(struct timer_list *t) { - struct rtc_device *rtc = from_timer(rtc, t, uie_timer); + struct rtc_device *rtc = timer_container_of(rtc, t, uie_timer); unsigned long flags; spin_lock_irqsave(&rtc->irq_lock, flags); diff --git a/drivers/rtc/rtc-test.c b/drivers/rtc/rtc-test.c index a9f5b9466fb5..94f995febe5b 100644 --- a/drivers/rtc/rtc-test.c +++ b/drivers/rtc/rtc-test.c @@ -107,7 +107,7 @@ static const struct rtc_class_ops test_rtc_ops = { static void test_rtc_alarm_handler(struct timer_list *t) { - struct rtc_test_data *rtd = from_timer(rtd, t, alarm); + struct rtc_test_data *rtd = timer_container_of(rtd, t, alarm); rtc_update_irq(rtd->rtc, 1, RTC_AF | RTC_IRQF); } diff --git a/drivers/s390/block/dasd.c b/drivers/s390/block/dasd.c index cf36d3bafeca..b16efecfde4b 100644 --- a/drivers/s390/block/dasd.c +++ b/drivers/s390/block/dasd.c @@ -1493,7 +1493,7 @@ static void dasd_device_timeout(struct timer_list *t) unsigned long flags; struct dasd_device *device; - device = from_timer(device, t, timer); + device = timer_container_of(device, t, timer); spin_lock_irqsave(get_ccwdev_lock(device->cdev), flags); /* re-activate request queue */ dasd_device_remove_stop_bits(device, DASD_STOPPED_PENDING); @@ -2677,7 +2677,7 @@ static void dasd_block_timeout(struct timer_list *t) unsigned long flags; struct dasd_block *block; - block = from_timer(block, t, timer); + block = timer_container_of(block, t, timer); spin_lock_irqsave(get_ccwdev_lock(block->base->cdev), flags); /* re-activate request queue */ dasd_device_remove_stop_bits(block->base, DASD_STOPPED_PENDING); diff --git a/drivers/s390/char/con3215.c b/drivers/s390/char/con3215.c index 6a61c0a595d9..56e43d43c713 100644 --- a/drivers/s390/char/con3215.c +++ b/drivers/s390/char/con3215.c @@ -284,7 +284,7 @@ static void raw3215_start_io(struct raw3215_info *raw) */ static void raw3215_timeout(struct timer_list *t) { - struct raw3215_info *raw = from_timer(raw, t, timer); + struct raw3215_info *raw = timer_container_of(raw, t, timer); unsigned long flags; spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags); diff --git a/drivers/s390/char/con3270.c b/drivers/s390/char/con3270.c index 8402a0042c0d..b78b86e8f281 100644 --- a/drivers/s390/char/con3270.c +++ b/drivers/s390/char/con3270.c @@ -525,7 +525,7 @@ static void tty3270_update_lines_all(struct tty3270 *tp, struct raw3270_request */ static void tty3270_update(struct timer_list *t) { - struct tty3270 *tp = from_timer(tp, t, timer); + struct tty3270 *tp = timer_container_of(tp, t, timer); struct raw3270_request *wrq; u8 cmd = TC_WRITE; int rc, len; diff --git a/drivers/s390/char/tape_core.c b/drivers/s390/char/tape_core.c index 48e8417a5cff..fa063e84eafc 100644 --- a/drivers/s390/char/tape_core.c +++ b/drivers/s390/char/tape_core.c @@ -821,7 +821,7 @@ tape_delayed_next_request(struct work_struct *work) static void tape_long_busy_timeout(struct timer_list *t) { - struct tape_device *device = from_timer(device, t, lb_timeout); + struct tape_device *device = timer_container_of(device, t, lb_timeout); struct tape_request *request; spin_lock_irq(get_ccwdev_lock(device->cdev)); diff --git a/drivers/s390/char/tape_std.c b/drivers/s390/char/tape_std.c index b76038632883..7ff177406bc3 100644 --- a/drivers/s390/char/tape_std.c +++ b/drivers/s390/char/tape_std.c @@ -35,7 +35,8 @@ static void tape_std_assign_timeout(struct timer_list *t) { - struct tape_request * request = from_timer(request, t, timer); + struct tape_request * request = timer_container_of(request, t, + timer); struct tape_device * device = request->device; int rc; diff --git a/drivers/s390/cio/device_fsm.c b/drivers/s390/cio/device_fsm.c index e1b1fbdabb1b..e849d3271b0e 100644 --- a/drivers/s390/cio/device_fsm.c +++ b/drivers/s390/cio/device_fsm.c @@ -98,7 +98,7 @@ static void ccw_timeout_log(struct ccw_device *cdev) void ccw_device_timeout(struct timer_list *t) { - struct ccw_device_private *priv = from_timer(priv, t, timer); + struct ccw_device_private *priv = timer_container_of(priv, t, timer); struct ccw_device *cdev = priv->cdev; spin_lock_irq(cdev->ccwlock); diff --git a/drivers/s390/cio/eadm_sch.c b/drivers/s390/cio/eadm_sch.c index ac382355dc04..37ea30be710c 100644 --- a/drivers/s390/cio/eadm_sch.c +++ b/drivers/s390/cio/eadm_sch.c @@ -98,7 +98,7 @@ static int eadm_subchannel_clear(struct subchannel *sch) static void eadm_subchannel_timeout(struct timer_list *t) { - struct eadm_private *private = from_timer(private, t, timer); + struct eadm_private *private = timer_container_of(private, t, timer); struct subchannel *sch = private->sch; spin_lock_irq(&sch->lock); diff --git a/drivers/s390/crypto/ap_bus.c b/drivers/s390/crypto/ap_bus.c index 288734cd8f4b..bd95bd390b5c 100644 --- a/drivers/s390/crypto/ap_bus.c +++ b/drivers/s390/crypto/ap_bus.c @@ -428,7 +428,7 @@ void ap_wait(enum ap_sm_wait wait) */ void ap_request_timeout(struct timer_list *t) { - struct ap_queue *aq = from_timer(aq, t, timeout); + struct ap_queue *aq = timer_container_of(aq, t, timeout); spin_lock_bh(&aq->lock); ap_wait(ap_sm_event(aq, AP_SM_EVENT_TIMEOUT)); diff --git a/drivers/s390/net/fsm.c b/drivers/s390/net/fsm.c index 5fcdce116862..6a12d2422540 100644 --- a/drivers/s390/net/fsm.c +++ b/drivers/s390/net/fsm.c @@ -132,7 +132,7 @@ fsm_getstate_str(fsm_instance *fi) static void fsm_expire_timer(struct timer_list *t) { - fsm_timer *this = from_timer(this, t, tl); + fsm_timer *this = timer_container_of(this, t, tl); #if FSM_TIMER_DEBUG printk(KERN_DEBUG "fsm(%s): Timer %p expired\n", this->fi->name, this); diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c index f5cfaebfb7c9..fe9f48c315d9 100644 --- a/drivers/s390/net/qeth_core_main.c +++ b/drivers/s390/net/qeth_core_main.c @@ -2619,7 +2619,8 @@ err_qdio_bufs: static void qeth_tx_completion_timer(struct timer_list *timer) { - struct qeth_qdio_out_q *queue = from_timer(queue, timer, timer); + struct qeth_qdio_out_q *queue = timer_container_of(queue, timer, + timer); napi_schedule(&queue->napi); QETH_TXQ_STAT_INC(queue, completion_timer); diff --git a/drivers/s390/scsi/zfcp_erp.c b/drivers/s390/scsi/zfcp_erp.c index 78d52a4c55f5..ffd994416995 100644 --- a/drivers/s390/scsi/zfcp_erp.c +++ b/drivers/s390/scsi/zfcp_erp.c @@ -615,7 +615,7 @@ void zfcp_erp_notify(struct zfcp_erp_action *erp_action, unsigned long set_mask) */ void zfcp_erp_timeout_handler(struct timer_list *t) { - struct zfcp_fsf_req *fsf_req = from_timer(fsf_req, t, timer); + struct zfcp_fsf_req *fsf_req = timer_container_of(fsf_req, t, timer); struct zfcp_erp_action *act; if (fsf_req->status & ZFCP_STATUS_FSFREQ_DISMISSED) @@ -629,7 +629,7 @@ void zfcp_erp_timeout_handler(struct timer_list *t) static void zfcp_erp_memwait_handler(struct timer_list *t) { - struct zfcp_erp_action *act = from_timer(act, t, timer); + struct zfcp_erp_action *act = timer_container_of(act, t, timer); zfcp_erp_notify(act, 0); } diff --git a/drivers/s390/scsi/zfcp_fsf.c b/drivers/s390/scsi/zfcp_fsf.c index d5f5f563881e..c5bba1be88f4 100644 --- a/drivers/s390/scsi/zfcp_fsf.c +++ b/drivers/s390/scsi/zfcp_fsf.c @@ -36,7 +36,7 @@ MODULE_PARM_DESC(ber_stop, static void zfcp_fsf_request_timeout_handler(struct timer_list *t) { - struct zfcp_fsf_req *fsf_req = from_timer(fsf_req, t, timer); + struct zfcp_fsf_req *fsf_req = timer_container_of(fsf_req, t, timer); struct zfcp_adapter *adapter = fsf_req->adapter; zfcp_qdio_siosl(adapter); diff --git a/drivers/s390/scsi/zfcp_qdio.c b/drivers/s390/scsi/zfcp_qdio.c index 0957e3f8b46e..f2410bc44ad3 100644 --- a/drivers/s390/scsi/zfcp_qdio.c +++ b/drivers/s390/scsi/zfcp_qdio.c @@ -102,7 +102,8 @@ static void zfcp_qdio_request_tasklet(struct tasklet_struct *tasklet) static void zfcp_qdio_request_timer(struct timer_list *timer) { - struct zfcp_qdio *qdio = from_timer(qdio, timer, request_timer); + struct zfcp_qdio *qdio = timer_container_of(qdio, timer, + request_timer); tasklet_schedule(&qdio->request_tasklet); } diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h index 8c384c25dca1..0a5888b53d6d 100644 --- a/drivers/scsi/aacraid/aacraid.h +++ b/drivers/scsi/aacraid/aacraid.h @@ -93,7 +93,6 @@ enum { #define AAC_NUM_MGT_FIB 8 #define AAC_NUM_IO_FIB (1024 - AAC_NUM_MGT_FIB) -#define AAC_NUM_FIB (AAC_NUM_IO_FIB + AAC_NUM_MGT_FIB) #define AAC_MAX_LUN 256 diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c index ffef61c4aa01..7d9a4dce236b 100644 --- a/drivers/scsi/aacraid/commsup.c +++ b/drivers/scsi/aacraid/commsup.c @@ -48,15 +48,7 @@ static int fib_map_alloc(struct aac_dev *dev) { - if (dev->max_fib_size > AAC_MAX_NATIVE_SIZE) - dev->max_cmd_size = AAC_MAX_NATIVE_SIZE; - else - dev->max_cmd_size = dev->max_fib_size; - if (dev->max_fib_size < AAC_MAX_NATIVE_SIZE) { - dev->max_cmd_size = AAC_MAX_NATIVE_SIZE; - } else { - dev->max_cmd_size = dev->max_fib_size; - } + dev->max_cmd_size = AAC_MAX_NATIVE_SIZE; dprintk((KERN_INFO "allocate hardware fibs dma_alloc_coherent(%p, %d * (%d + %d), %p)\n", diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c index f9372a81cd4e..6b87ea004e53 100644 --- a/drivers/scsi/aic7xxx/aic79xx_core.c +++ b/drivers/scsi/aic7xxx/aic79xx_core.c @@ -8784,7 +8784,7 @@ ahd_reset_channel(struct ahd_softc *ahd, char channel, int initiate_reset) static void ahd_stat_timer(struct timer_list *t) { - struct ahd_softc *ahd = from_timer(ahd, t, stat_timer); + struct ahd_softc *ahd = timer_container_of(ahd, t, stat_timer); u_long s; int enint_coal; diff --git a/drivers/scsi/aic94xx/aic94xx_scb.c b/drivers/scsi/aic94xx/aic94xx_scb.c index 68214a58b160..08c8dad9ad62 100644 --- a/drivers/scsi/aic94xx/aic94xx_scb.c +++ b/drivers/scsi/aic94xx/aic94xx_scb.c @@ -862,7 +862,7 @@ void asd_build_initiate_link_adm_task(struct asd_ascb *ascb, int phy_id, */ void asd_ascb_timedout(struct timer_list *t) { - struct asd_ascb *ascb = from_timer(ascb, t, timer); + struct asd_ascb *ascb = timer_container_of(ascb, t, timer); struct asd_seq_data *seq = &ascb->ha->seq; unsigned long flags; diff --git a/drivers/scsi/aic94xx/aic94xx_tmf.c b/drivers/scsi/aic94xx/aic94xx_tmf.c index d45dbf98f25e..28ac92b041fe 100644 --- a/drivers/scsi/aic94xx/aic94xx_tmf.c +++ b/drivers/scsi/aic94xx/aic94xx_tmf.c @@ -70,7 +70,7 @@ static void asd_clear_nexus_tasklet_complete(struct asd_ascb *ascb, static void asd_clear_nexus_timedout(struct timer_list *t) { - struct asd_ascb *ascb = from_timer(ascb, t, timer); + struct asd_ascb *ascb = timer_container_of(ascb, t, timer); struct tasklet_completion_status *tcs = ascb->uldd_task; ASD_DPRINTK("%s: here\n", __func__); @@ -244,7 +244,7 @@ static int asd_clear_nexus_index(struct sas_task *task) static void asd_tmf_timedout(struct timer_list *t) { - struct asd_ascb *ascb = from_timer(ascb, t, timer); + struct asd_ascb *ascb = timer_container_of(ascb, t, timer); struct tasklet_completion_status *tcs = ascb->uldd_task; ASD_DPRINTK("tmf timed out\n"); diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c index b450b1fc6bbb..fb57343a97bd 100644 --- a/drivers/scsi/arcmsr/arcmsr_hba.c +++ b/drivers/scsi/arcmsr/arcmsr_hba.c @@ -3935,7 +3935,8 @@ static int arcmsr_polling_ccbdone(struct AdapterControlBlock *acb, static void arcmsr_set_iop_datetime(struct timer_list *t) { - struct AdapterControlBlock *pacb = from_timer(pacb, t, refresh_timer); + struct AdapterControlBlock *pacb = timer_container_of(pacb, t, + refresh_timer); unsigned int next_time; struct tm tm; @@ -4263,7 +4264,8 @@ static void arcmsr_wait_firmware_ready(struct AdapterControlBlock *acb) static void arcmsr_request_device_map(struct timer_list *t) { - struct AdapterControlBlock *acb = from_timer(acb, t, eternal_timer); + struct AdapterControlBlock *acb = timer_container_of(acb, t, + eternal_timer); if (acb->acb_flags & (ACB_F_MSG_GET_CONFIG | ACB_F_BUS_RESET | ACB_F_ABORT)) { mod_timer(&acb->eternal_timer, jiffies + msecs_to_jiffies(6 * HZ)); } else { diff --git a/drivers/scsi/arm/fas216.c b/drivers/scsi/arm/fas216.c index e0b55d869a35..b1a749ab18f8 100644 --- a/drivers/scsi/arm/fas216.c +++ b/drivers/scsi/arm/fas216.c @@ -2327,7 +2327,7 @@ DEF_SCSI_QCMD(fas216_noqueue_command) */ static void fas216_eh_timer(struct timer_list *t) { - FAS216_Info *info = from_timer(info, t, eh_timer); + FAS216_Info *info = timer_container_of(info, t, eh_timer); fas216_log(info, LOG_ERROR, "error handling timed out\n"); diff --git a/drivers/scsi/be2iscsi/be_main.c b/drivers/scsi/be2iscsi/be_main.c index 7d1b767d87fb..dc88bc46dcc0 100644 --- a/drivers/scsi/be2iscsi/be_main.c +++ b/drivers/scsi/be2iscsi/be_main.c @@ -5240,7 +5240,7 @@ static void beiscsi_eqd_update_work(struct work_struct *work) static void beiscsi_hw_tpe_check(struct timer_list *t) { - struct beiscsi_hba *phba = from_timer(phba, t, hw_check); + struct beiscsi_hba *phba = timer_container_of(phba, t, hw_check); u32 wait; /* if not TPE, do nothing */ @@ -5257,7 +5257,7 @@ static void beiscsi_hw_tpe_check(struct timer_list *t) static void beiscsi_hw_health_check(struct timer_list *t) { - struct beiscsi_hba *phba = from_timer(phba, t, hw_check); + struct beiscsi_hba *phba = timer_container_of(phba, t, hw_check); beiscsi_detect_ue(phba); if (beiscsi_detect_ue(phba)) { diff --git a/drivers/scsi/bfa/bfad.c b/drivers/scsi/bfa/bfad.c index 598f2fc93ef2..ff9adfc0b332 100644 --- a/drivers/scsi/bfa/bfad.c +++ b/drivers/scsi/bfa/bfad.c @@ -685,7 +685,8 @@ ext: void bfad_bfa_tmo(struct timer_list *t) { - struct bfad_s *bfad = from_timer(bfad, t, hal_tmo); + struct bfad_s *bfad = timer_container_of(bfad, t, + hal_tmo); unsigned long flags; struct list_head doneq; diff --git a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c index de6574cccf58..58da993251e9 100644 --- a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c +++ b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c @@ -837,7 +837,7 @@ static int bnx2fc_net_config(struct fc_lport *lport, struct net_device *netdev) static void bnx2fc_destroy_timer(struct timer_list *t) { - struct bnx2fc_hba *hba = from_timer(hba, t, destroy_timer); + struct bnx2fc_hba *hba = timer_container_of(hba, t, destroy_timer); printk(KERN_ERR PFX "ERROR:bnx2fc_destroy_timer - " "Destroy compl not received!!\n"); diff --git a/drivers/scsi/bnx2fc/bnx2fc_tgt.c b/drivers/scsi/bnx2fc/bnx2fc_tgt.c index b8227cfef64f..77dcdfc412b1 100644 --- a/drivers/scsi/bnx2fc/bnx2fc_tgt.c +++ b/drivers/scsi/bnx2fc/bnx2fc_tgt.c @@ -30,7 +30,7 @@ static void bnx2fc_free_conn_id(struct bnx2fc_hba *hba, u32 conn_id); static void bnx2fc_upld_timer(struct timer_list *t) { - struct bnx2fc_rport *tgt = from_timer(tgt, t, upld_timer); + struct bnx2fc_rport *tgt = timer_container_of(tgt, t, upld_timer); BNX2FC_TGT_DBG(tgt, "upld_timer - Upload compl not received!!\n"); /* fake upload completion */ @@ -43,7 +43,7 @@ static void bnx2fc_upld_timer(struct timer_list *t) static void bnx2fc_ofld_timer(struct timer_list *t) { - struct bnx2fc_rport *tgt = from_timer(tgt, t, ofld_timer); + struct bnx2fc_rport *tgt = timer_container_of(tgt, t, ofld_timer); BNX2FC_TGT_DBG(tgt, "entered bnx2fc_ofld_timer\n"); /* NOTE: This function should never be called, as diff --git a/drivers/scsi/bnx2i/bnx2i_hwi.c b/drivers/scsi/bnx2i/bnx2i_hwi.c index 6c864b093ac9..40db5190a222 100644 --- a/drivers/scsi/bnx2i/bnx2i_hwi.c +++ b/drivers/scsi/bnx2i/bnx2i_hwi.c @@ -685,7 +685,7 @@ void bnx2i_update_iscsi_conn(struct iscsi_conn *conn) */ void bnx2i_ep_ofld_timer(struct timer_list *t) { - struct bnx2i_endpoint *ep = from_timer(ep, t, ofld_timer); + struct bnx2i_endpoint *ep = timer_container_of(ep, t, ofld_timer); if (ep->state == EP_STATE_OFLD_START) { printk(KERN_ALERT "ofld_timer: CONN_OFLD timeout\n"); diff --git a/drivers/scsi/csiostor/csio_hw.c b/drivers/scsi/csiostor/csio_hw.c index beded091dff1..7aa418ebfe01 100644 --- a/drivers/scsi/csiostor/csio_hw.c +++ b/drivers/scsi/csiostor/csio_hw.c @@ -3738,7 +3738,7 @@ csio_mberr_worker(void *data) static void csio_hw_mb_timer(struct timer_list *t) { - struct csio_mbm *mbm = from_timer(mbm, t, timer); + struct csio_mbm *mbm = timer_container_of(mbm, t, timer); struct csio_hw *hw = mbm->hw; struct csio_mb *mbp = NULL; @@ -4107,7 +4107,7 @@ csio_mgmt_req_lookup(struct csio_mgmtm *mgmtm, struct csio_ioreq *io_req) static void csio_mgmt_tmo_handler(struct timer_list *t) { - struct csio_mgmtm *mgmtm = from_timer(mgmtm, t, mgmt_timer); + struct csio_mgmtm *mgmtm = timer_container_of(mgmtm, t, mgmt_timer); struct list_head *tmp; struct csio_ioreq *io_req; diff --git a/drivers/scsi/cxgbi/cxgb3i/cxgb3i.c b/drivers/scsi/cxgbi/cxgb3i/cxgb3i.c index 461d38e2fb19..69de9657f7cb 100644 --- a/drivers/scsi/cxgbi/cxgb3i/cxgb3i.c +++ b/drivers/scsi/cxgbi/cxgb3i/cxgb3i.c @@ -547,7 +547,7 @@ static int act_open_rpl_status_to_errno(int status) static void act_open_retry_timer(struct timer_list *t) { - struct cxgbi_sock *csk = from_timer(csk, t, retry_timer); + struct cxgbi_sock *csk = timer_container_of(csk, t, retry_timer); struct sk_buff *skb; log_debug(1 << CXGBI_DBG_TOE | 1 << CXGBI_DBG_SOCK, diff --git a/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c b/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c index aaba294ecb58..42676627c3af 100644 --- a/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c +++ b/drivers/scsi/cxgbi/cxgb4i/cxgb4i.c @@ -988,7 +988,7 @@ static int act_open_rpl_status_to_errno(int status) static void csk_act_open_retry_timer(struct timer_list *t) { struct sk_buff *skb = NULL; - struct cxgbi_sock *csk = from_timer(csk, t, retry_timer); + struct cxgbi_sock *csk = timer_container_of(csk, t, retry_timer); struct cxgb4_lld_info *lldi = cxgbi_cdev_priv(csk->cdev); void (*send_act_open_func)(struct cxgbi_sock *, struct sk_buff *, struct l2t_entry *); diff --git a/drivers/scsi/dc395x.c b/drivers/scsi/dc395x.c index 96b335c92603..386c8359e1cc 100644 --- a/drivers/scsi/dc395x.c +++ b/drivers/scsi/dc395x.c @@ -751,7 +751,7 @@ static void waiting_process_next(struct AdapterCtlBlk *acb) static void waiting_timeout(struct timer_list *t) { unsigned long flags; - struct AdapterCtlBlk *acb = from_timer(acb, t, waiting_timer); + struct AdapterCtlBlk *acb = timer_container_of(acb, t, waiting_timer); DC395x_LOCK_IO(acb->scsi_host, flags); waiting_process_next(acb); DC395x_UNLOCK_IO(acb->scsi_host, flags); diff --git a/drivers/scsi/elx/efct/efct_xport.c b/drivers/scsi/elx/efct/efct_xport.c index 2aca60f6428e..dfe05fab7b42 100644 --- a/drivers/scsi/elx/efct/efct_xport.c +++ b/drivers/scsi/elx/efct/efct_xport.c @@ -180,7 +180,7 @@ efct_xport_config_stats_timer(struct efct *efct); static void efct_xport_stats_timer_cb(struct timer_list *t) { - struct efct_xport *xport = from_timer(xport, t, stats_timer); + struct efct_xport *xport = timer_container_of(xport, t, stats_timer); struct efct *efct = xport->efct; efct_xport_config_stats_timer(efct); diff --git a/drivers/scsi/elx/libefc/efc_els.c b/drivers/scsi/elx/libefc/efc_els.c index 84bc81d7ce76..1786cee08729 100644 --- a/drivers/scsi/elx/libefc/efc_els.c +++ b/drivers/scsi/elx/libefc/efc_els.c @@ -147,7 +147,7 @@ efc_els_retry(struct efc_els_io_req *els); static void efc_els_delay_timer_cb(struct timer_list *t) { - struct efc_els_io_req *els = from_timer(els, t, delay_timer); + struct efc_els_io_req *els = timer_container_of(els, t, delay_timer); /* Retry delay timer expired, retry the ELS request */ efc_els_retry(els); diff --git a/drivers/scsi/elx/libefc/efc_fabric.c b/drivers/scsi/elx/libefc/efc_fabric.c index cf7e738c4edc..4ed9f46ded65 100644 --- a/drivers/scsi/elx/libefc/efc_fabric.c +++ b/drivers/scsi/elx/libefc/efc_fabric.c @@ -886,7 +886,7 @@ __efc_ns_idle(struct efc_sm_ctx *ctx, enum efc_sm_event evt, void *arg) static void gidpt_delay_timer_cb(struct timer_list *t) { - struct efc_node *node = from_timer(node, t, gidpt_delay_timer); + struct efc_node *node = timer_container_of(node, t, gidpt_delay_timer); timer_delete(&node->gidpt_delay_timer); diff --git a/drivers/scsi/esas2r/esas2r_main.c b/drivers/scsi/esas2r/esas2r_main.c index 44871746944a..3f31875ff46e 100644 --- a/drivers/scsi/esas2r/esas2r_main.c +++ b/drivers/scsi/esas2r/esas2r_main.c @@ -1585,7 +1585,7 @@ void esas2r_kickoff_timer(struct esas2r_adapter *a) static void esas2r_timer_callback(struct timer_list *t) { - struct esas2r_adapter *a = from_timer(a, t, timer); + struct esas2r_adapter *a = timer_container_of(a, t, timer); set_bit(AF2_TIMER_TICK, &a->flags2); diff --git a/drivers/scsi/fcoe/fcoe_ctlr.c b/drivers/scsi/fcoe/fcoe_ctlr.c index 56d270526c9c..8e4241c295e3 100644 --- a/drivers/scsi/fcoe/fcoe_ctlr.c +++ b/drivers/scsi/fcoe/fcoe_ctlr.c @@ -1773,7 +1773,7 @@ unlock: */ static void fcoe_ctlr_timeout(struct timer_list *t) { - struct fcoe_ctlr *fip = from_timer(fip, t, timer); + struct fcoe_ctlr *fip = timer_container_of(fip, t, timer); schedule_work(&fip->timer_work); } diff --git a/drivers/scsi/fcoe/fcoe_transport.c b/drivers/scsi/fcoe/fcoe_transport.c index a48d24af9ac3..2f478426f16e 100644 --- a/drivers/scsi/fcoe/fcoe_transport.c +++ b/drivers/scsi/fcoe/fcoe_transport.c @@ -447,7 +447,7 @@ EXPORT_SYMBOL_GPL(fcoe_check_wait_queue); */ void fcoe_queue_timer(struct timer_list *t) { - struct fcoe_port *port = from_timer(port, t, timer); + struct fcoe_port *port = timer_container_of(port, t, timer); fcoe_check_wait_queue(port->lport, NULL); } diff --git a/drivers/scsi/fnic/fdls_disc.c b/drivers/scsi/fnic/fdls_disc.c index c2b6f4eb338e..f8ab69c51dab 100644 --- a/drivers/scsi/fnic/fdls_disc.c +++ b/drivers/scsi/fnic/fdls_disc.c @@ -2074,7 +2074,8 @@ static void fdls_fdmi_register_pa(struct fnic_iport_s *iport) void fdls_fabric_timer_callback(struct timer_list *t) { - struct fnic_fdls_fabric_s *fabric = from_timer(fabric, t, retry_timer); + struct fnic_fdls_fabric_s *fabric = timer_container_of(fabric, t, + retry_timer); struct fnic_iport_s *iport = container_of(fabric, struct fnic_iport_s, fabric); struct fnic *fnic = iport->fnic; @@ -2246,7 +2247,8 @@ void fdls_fabric_timer_callback(struct timer_list *t) void fdls_fdmi_timer_callback(struct timer_list *t) { - struct fnic_fdls_fabric_s *fabric = from_timer(fabric, t, fdmi_timer); + struct fnic_fdls_fabric_s *fabric = timer_container_of(fabric, t, + fdmi_timer); struct fnic_iport_s *iport = container_of(fabric, struct fnic_iport_s, fabric); struct fnic *fnic = iport->fnic; @@ -2323,7 +2325,7 @@ static void fdls_send_delete_tport_msg(struct fnic_tport_s *tport) static void fdls_tport_timer_callback(struct timer_list *t) { - struct fnic_tport_s *tport = from_timer(tport, t, retry_timer); + struct fnic_tport_s *tport = timer_container_of(tport, t, retry_timer); struct fnic_iport_s *iport = (struct fnic_iport_s *) tport->iport; struct fnic *fnic = iport->fnic; uint16_t oxid; diff --git a/drivers/scsi/fnic/fip.c b/drivers/scsi/fnic/fip.c index 19395e2aee44..ce62ab1180bd 100644 --- a/drivers/scsi/fnic/fip.c +++ b/drivers/scsi/fnic/fip.c @@ -777,7 +777,7 @@ void fnic_work_on_fip_timer(struct work_struct *work) */ void fnic_handle_fip_timer(struct timer_list *t) { - struct fnic *fnic = from_timer(fnic, t, retry_fip_timer); + struct fnic *fnic = timer_container_of(fnic, t, retry_fip_timer); INIT_WORK(&fnic->fip_timer_work, fnic_work_on_fip_timer); queue_work(fnic_fip_queue, &fnic->fip_timer_work); @@ -790,7 +790,7 @@ void fnic_handle_fip_timer(struct timer_list *t) void fnic_handle_enode_ka_timer(struct timer_list *t) { uint8_t *frame; - struct fnic *fnic = from_timer(fnic, t, enode_ka_timer); + struct fnic *fnic = timer_container_of(fnic, t, enode_ka_timer); struct fnic_iport_s *iport = &fnic->iport; struct fip_enode_ka *penode_ka; @@ -843,7 +843,7 @@ void fnic_handle_enode_ka_timer(struct timer_list *t) void fnic_handle_vn_ka_timer(struct timer_list *t) { uint8_t *frame; - struct fnic *fnic = from_timer(fnic, t, vn_ka_timer); + struct fnic *fnic = timer_container_of(fnic, t, vn_ka_timer); struct fnic_iport_s *iport = &fnic->iport; struct fip_vn_port_ka *pvn_port_ka; @@ -998,7 +998,7 @@ void fnic_work_on_fcs_ka_timer(struct work_struct *work) */ void fnic_handle_fcs_ka_timer(struct timer_list *t) { - struct fnic *fnic = from_timer(fnic, t, fcs_ka_timer); + struct fnic *fnic = timer_container_of(fnic, t, fcs_ka_timer); INIT_WORK(&fnic->fip_timer_work, fnic_work_on_fcs_ka_timer); queue_work(fnic_fip_queue, &fnic->fip_timer_work); diff --git a/drivers/scsi/fnic/fnic_main.c b/drivers/scsi/fnic/fnic_main.c index 9a357ff42085..4cc4077ea53c 100644 --- a/drivers/scsi/fnic/fnic_main.c +++ b/drivers/scsi/fnic/fnic_main.c @@ -446,7 +446,7 @@ static int fnic_notify_set(struct fnic *fnic) static void fnic_notify_timer(struct timer_list *t) { - struct fnic *fnic = from_timer(fnic, t, notify_timer); + struct fnic *fnic = timer_container_of(fnic, t, notify_timer); fnic_handle_link_event(fnic); mod_timer(&fnic->notify_timer, diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 4864e957be0b..d1a4cc69d408 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -1016,7 +1016,7 @@ EXPORT_SYMBOL_GPL(hisi_sas_notify_phy_event); static void hisi_sas_wait_phyup_timedout(struct timer_list *t) { - struct hisi_sas_phy *phy = from_timer(phy, t, timer); + struct hisi_sas_phy *phy = timer_container_of(phy, t, timer); struct hisi_hba *hisi_hba = phy->hisi_hba; struct device *dev = hisi_hba->dev; int phy_no = phy->sas_phy.id; diff --git a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c index 6d97339371fb..fa94d7110714 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c @@ -795,7 +795,7 @@ static void phy_hard_reset_v1_hw(struct hisi_hba *hisi_hba, int phy_no) static void start_phys_v1_hw(struct timer_list *t) { - struct hisi_hba *hisi_hba = from_timer(hisi_hba, t, timer); + struct hisi_hba *hisi_hba = timer_container_of(hisi_hba, t, timer); int i; for (i = 0; i < hisi_hba->n_phy; i++) { diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c index 2adfedb8484c..24cd172905f3 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c @@ -1328,7 +1328,7 @@ static void init_reg_v2_hw(struct hisi_hba *hisi_hba) static void link_timeout_enable_link(struct timer_list *t) { - struct hisi_hba *hisi_hba = from_timer(hisi_hba, t, timer); + struct hisi_hba *hisi_hba = timer_container_of(hisi_hba, t, timer); int i, reg_val; for (i = 0; i < hisi_hba->n_phy; i++) { @@ -1349,7 +1349,7 @@ static void link_timeout_enable_link(struct timer_list *t) static void link_timeout_disable_link(struct timer_list *t) { - struct hisi_hba *hisi_hba = from_timer(hisi_hba, t, timer); + struct hisi_hba *hisi_hba = timer_container_of(hisi_hba, t, timer); int i, reg_val; reg_val = hisi_sas_read32(hisi_hba, PHY_STATE); @@ -2581,7 +2581,8 @@ static void prep_ata_v2_hw(struct hisi_hba *hisi_hba, static void hisi_sas_internal_abort_quirk_timeout(struct timer_list *t) { - struct hisi_sas_slot *slot = from_timer(slot, t, internal_abort_timer); + struct hisi_sas_slot *slot = timer_container_of(slot, t, + internal_abort_timer); struct hisi_sas_port *port = slot->port; struct asd_sas_port *asd_sas_port; struct asd_sas_phy *sas_phy; diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 4c493b06062a..862ab0fbc893 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1697,7 +1697,7 @@ static int ibmvfc_map_sg_data(struct scsi_cmnd *scmd, **/ static void ibmvfc_timeout(struct timer_list *t) { - struct ibmvfc_event *evt = from_timer(evt, t, timer); + struct ibmvfc_event *evt = timer_container_of(evt, t, timer); struct ibmvfc_host *vhost = evt->vhost; dev_err(vhost->dev, "Command timed out (%p). Resetting connection\n", evt); ibmvfc_reset_host(vhost); @@ -4630,7 +4630,7 @@ static void ibmvfc_tgt_adisc_cancel_done(struct ibmvfc_event *evt) **/ static void ibmvfc_adisc_timeout(struct timer_list *t) { - struct ibmvfc_target *tgt = from_timer(tgt, t, timer); + struct ibmvfc_target *tgt = timer_container_of(tgt, t, timer); struct ibmvfc_host *vhost = tgt->vhost; struct ibmvfc_event *evt; struct ibmvfc_tmf *tmf; diff --git a/drivers/scsi/ibmvscsi/ibmvscsi.c b/drivers/scsi/ibmvscsi/ibmvscsi.c index d65a45860b33..3d65a498b701 100644 --- a/drivers/scsi/ibmvscsi/ibmvscsi.c +++ b/drivers/scsi/ibmvscsi/ibmvscsi.c @@ -845,7 +845,8 @@ static void ibmvscsi_reset_host(struct ibmvscsi_host_data *hostdata) */ static void ibmvscsi_timeout(struct timer_list *t) { - struct srp_event_struct *evt_struct = from_timer(evt_struct, t, timer); + struct srp_event_struct *evt_struct = timer_container_of(evt_struct, + t, timer); struct ibmvscsi_host_data *hostdata = evt_struct->hostdata; dev_err(hostdata->dev, "Command timed out (%x). Resetting connection\n", diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c index d89135fb8faa..b29bec6abd72 100644 --- a/drivers/scsi/ipr.c +++ b/drivers/scsi/ipr.c @@ -2589,7 +2589,7 @@ static void ipr_process_error(struct ipr_cmnd *ipr_cmd) **/ static void ipr_timeout(struct timer_list *t) { - struct ipr_cmnd *ipr_cmd = from_timer(ipr_cmd, t, timer); + struct ipr_cmnd *ipr_cmd = timer_container_of(ipr_cmd, t, timer); unsigned long lock_flags = 0; struct ipr_ioa_cfg *ioa_cfg = ipr_cmd->ioa_cfg; @@ -2622,7 +2622,7 @@ static void ipr_timeout(struct timer_list *t) **/ static void ipr_oper_timeout(struct timer_list *t) { - struct ipr_cmnd *ipr_cmd = from_timer(ipr_cmd, t, timer); + struct ipr_cmnd *ipr_cmd = timer_container_of(ipr_cmd, t, timer); unsigned long lock_flags = 0; struct ipr_ioa_cfg *ioa_cfg = ipr_cmd->ioa_cfg; @@ -5151,7 +5151,7 @@ static void ipr_bus_reset_done(struct ipr_cmnd *ipr_cmd) **/ static void ipr_abort_timeout(struct timer_list *t) { - struct ipr_cmnd *ipr_cmd = from_timer(ipr_cmd, t, timer); + struct ipr_cmnd *ipr_cmd = timer_container_of(ipr_cmd, t, timer); struct ipr_cmnd *reset_cmd; struct ipr_ioa_cfg *ioa_cfg = ipr_cmd->ioa_cfg; struct ipr_cmd_pkt *cmd_pkt; @@ -7476,7 +7476,7 @@ static int ipr_ioafp_identify_hrrq(struct ipr_cmnd *ipr_cmd) **/ static void ipr_reset_timer_done(struct timer_list *t) { - struct ipr_cmnd *ipr_cmd = from_timer(ipr_cmd, t, timer); + struct ipr_cmnd *ipr_cmd = timer_container_of(ipr_cmd, t, timer); struct ipr_ioa_cfg *ioa_cfg = ipr_cmd->ioa_cfg; unsigned long lock_flags = 0; diff --git a/drivers/scsi/isci/host.c b/drivers/scsi/isci/host.c index c108b5b940c3..6d2f4c831df7 100644 --- a/drivers/scsi/isci/host.c +++ b/drivers/scsi/isci/host.c @@ -958,7 +958,7 @@ static enum sci_status sci_controller_start_next_phy(struct isci_host *ihost) static void phy_startup_timeout(struct timer_list *t) { - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct isci_host *ihost = container_of(tmr, typeof(*ihost), phy_timer); unsigned long flags; enum sci_status status; @@ -1592,7 +1592,7 @@ static const struct sci_base_state sci_controller_state_table[] = { static void controller_timeout(struct timer_list *t) { - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct isci_host *ihost = container_of(tmr, typeof(*ihost), timer); struct sci_base_state_machine *sm = &ihost->sm; unsigned long flags; @@ -1737,7 +1737,7 @@ static u8 max_spin_up(struct isci_host *ihost) static void power_control_timeout(struct timer_list *t) { - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct isci_host *ihost = container_of(tmr, typeof(*ihost), power_control.timer); struct isci_phy *iphy; unsigned long flags; diff --git a/drivers/scsi/isci/phy.c b/drivers/scsi/isci/phy.c index 743a3c64b0da..88237ec8b15f 100644 --- a/drivers/scsi/isci/phy.c +++ b/drivers/scsi/isci/phy.c @@ -317,7 +317,7 @@ sci_phy_link_layer_initialization(struct isci_phy *iphy, static void phy_sata_timeout(struct timer_list *t) { - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct isci_phy *iphy = container_of(tmr, typeof(*iphy), sata_timer); struct isci_host *ihost = iphy->owning_port->owning_controller; unsigned long flags; diff --git a/drivers/scsi/isci/port.c b/drivers/scsi/isci/port.c index 1609aba1c9c1..10bd2aac2cb4 100644 --- a/drivers/scsi/isci/port.c +++ b/drivers/scsi/isci/port.c @@ -775,7 +775,7 @@ bool sci_port_link_detected(struct isci_port *iport, struct isci_phy *iphy) static void port_timeout(struct timer_list *t) { - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct isci_port *iport = container_of(tmr, typeof(*iport), timer); struct isci_host *ihost = iport->owning_controller; unsigned long flags; diff --git a/drivers/scsi/isci/port_config.c b/drivers/scsi/isci/port_config.c index c382a257b51b..3b4820defe63 100644 --- a/drivers/scsi/isci/port_config.c +++ b/drivers/scsi/isci/port_config.c @@ -321,7 +321,7 @@ sci_mpc_agent_validate_phy_configuration(struct isci_host *ihost, static void mpc_agent_timeout(struct timer_list *t) { u8 index; - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct sci_port_configuration_agent *port_agent; struct isci_host *ihost; unsigned long flags; @@ -659,7 +659,7 @@ static void sci_apc_agent_link_down( static void apc_agent_timeout(struct timer_list *t) { u32 index; - struct sci_timer *tmr = from_timer(tmr, t, timer); + struct sci_timer *tmr = timer_container_of(tmr, t, timer); struct sci_port_configuration_agent *port_agent; struct isci_host *ihost; unsigned long flags; diff --git a/drivers/scsi/libfc/fc_fcp.c b/drivers/scsi/libfc/fc_fcp.c index e705c30b4e1b..16d0f02af1e4 100644 --- a/drivers/scsi/libfc/fc_fcp.c +++ b/drivers/scsi/libfc/fc_fcp.c @@ -1283,7 +1283,7 @@ static int fc_fcp_pkt_abort(struct fc_fcp_pkt *fsp) */ static void fc_lun_reset_send(struct timer_list *t) { - struct fc_fcp_pkt *fsp = from_timer(fsp, t, timer); + struct fc_fcp_pkt *fsp = timer_container_of(fsp, t, timer); struct fc_lport *lport = fsp->lp; if (lport->tt.fcp_cmd_send(lport, fsp, fc_tm_done)) { @@ -1416,7 +1416,7 @@ static void fc_fcp_cleanup(struct fc_lport *lport) */ static void fc_fcp_timeout(struct timer_list *t) { - struct fc_fcp_pkt *fsp = from_timer(fsp, t, timer); + struct fc_fcp_pkt *fsp = timer_container_of(fsp, t, timer); struct fc_rport *rport = fsp->rport; struct fc_rport_libfc_priv *rpriv = rport->dd_data; diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c index 1ddaf7228340..392d57e054db 100644 --- a/drivers/scsi/libiscsi.c +++ b/drivers/scsi/libiscsi.c @@ -1898,7 +1898,8 @@ EXPORT_SYMBOL_GPL(iscsi_target_alloc); static void iscsi_tmf_timedout(struct timer_list *t) { - struct iscsi_session *session = from_timer(session, t, tmf_timer); + struct iscsi_session *session = timer_container_of(session, t, + tmf_timer); spin_lock(&session->frwd_lock); if (session->tmf_state == TMF_QUEUED) { @@ -2240,7 +2241,7 @@ EXPORT_SYMBOL_GPL(iscsi_eh_cmd_timed_out); static void iscsi_check_transport_timeouts(struct timer_list *t) { - struct iscsi_conn *conn = from_timer(conn, t, transport_timer); + struct iscsi_conn *conn = timer_container_of(conn, t, transport_timer); struct iscsi_session *session = conn->session; unsigned long recv_timeout, next_timeout = 0, last_recv; diff --git a/drivers/scsi/libsas/sas_scsi_host.c b/drivers/scsi/libsas/sas_scsi_host.c index feb2461b90e8..928723c90b75 100644 --- a/drivers/scsi/libsas/sas_scsi_host.c +++ b/drivers/scsi/libsas/sas_scsi_host.c @@ -865,7 +865,7 @@ void sas_task_internal_done(struct sas_task *task) void sas_task_internal_timedout(struct timer_list *t) { - struct sas_task_slow *slow = from_timer(slow, t, timer); + struct sas_task_slow *slow = timer_container_of(slow, t, timer); struct sas_task *task = slow->task; bool is_completed = true; unsigned long flags; diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c index 12c67cdd7c19..530dddd39bab 100644 --- a/drivers/scsi/lpfc/lpfc_ct.c +++ b/drivers/scsi/lpfc/lpfc_ct.c @@ -3433,7 +3433,8 @@ fdmi_cmd_exit: void lpfc_delayed_disc_tmo(struct timer_list *t) { - struct lpfc_vport *vport = from_timer(vport, t, delayed_disc_tmo); + struct lpfc_vport *vport = timer_container_of(vport, t, + delayed_disc_tmo); struct lpfc_hba *phba = vport->phba; uint32_t tmo_posted; unsigned long iflag; diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 375a879c31f1..b1a61eca8295 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -4378,7 +4378,8 @@ lpfc_cancel_retry_delay_tmo(struct lpfc_vport *vport, struct lpfc_nodelist *nlp) void lpfc_els_retry_delay(struct timer_list *t) { - struct lpfc_nodelist *ndlp = from_timer(ndlp, t, nlp_delayfunc); + struct lpfc_nodelist *ndlp = timer_container_of(ndlp, t, + nlp_delayfunc); struct lpfc_vport *vport = ndlp->vport; struct lpfc_hba *phba = vport->phba; unsigned long flags; @@ -9385,7 +9386,7 @@ out: void lpfc_els_timeout(struct timer_list *t) { - struct lpfc_vport *vport = from_timer(vport, t, els_tmofunc); + struct lpfc_vport *vport = timer_container_of(vport, t, els_tmofunc); struct lpfc_hba *phba = vport->phba; uint32_t tmo_posted; unsigned long iflag; @@ -11594,7 +11595,8 @@ err: void lpfc_fabric_block_timeout(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, fabric_block_timer); + struct lpfc_hba *phba = timer_container_of(phba, t, + fabric_block_timer); unsigned long iflags; uint32_t tmo_posted; diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 3d15a964f5c9..b88e54a7e65c 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -6059,7 +6059,7 @@ lpfc_cleanup_discovery_resources(struct lpfc_vport *vport) void lpfc_disc_timeout(struct timer_list *t) { - struct lpfc_vport *vport = from_timer(vport, t, fc_disctmo); + struct lpfc_vport *vport = timer_container_of(vport, t, fc_disctmo); struct lpfc_hba *phba = vport->phba; uint32_t tmo_posted; unsigned long flags = 0; diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 2400602a8561..20fa450def03 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -1196,7 +1196,7 @@ lpfc_hb_timeout(struct timer_list *t) uint32_t tmo_posted; unsigned long iflag; - phba = from_timer(phba, t, hb_tmofunc); + phba = timer_container_of(phba, t, hb_tmofunc); /* Check for heart beat timeout conditions */ spin_lock_irqsave(&phba->pport->work_port_lock, iflag); @@ -1228,7 +1228,7 @@ lpfc_rrq_timeout(struct timer_list *t) { struct lpfc_hba *phba; - phba = from_timer(phba, t, rrq_tmr); + phba = timer_container_of(phba, t, rrq_tmr); if (test_bit(FC_UNLOADING, &phba->pport->load_flag)) { clear_bit(HBA_RRQ_ACTIVE, &phba->hba_flag); return; @@ -5131,7 +5131,7 @@ lpfc_fcf_redisc_wait_start_timer(struct lpfc_hba *phba) static void lpfc_sli4_fcf_redisc_wait_tmo(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, fcf.redisc_wait); + struct lpfc_hba *phba = timer_container_of(phba, t, fcf.redisc_wait); /* Don't send FCF rediscovery event if timer cancelled */ spin_lock_irq(&phba->hbalock); @@ -5162,7 +5162,8 @@ lpfc_sli4_fcf_redisc_wait_tmo(struct timer_list *t) static void lpfc_vmid_poll(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, inactive_vmid_poll); + struct lpfc_hba *phba = timer_container_of(phba, t, + inactive_vmid_poll); u32 wake_up = 0; /* check if there is a need to issue QFPA */ diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index 9edf80b14b1a..8acb744febcd 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -5190,7 +5190,7 @@ void lpfc_poll_start_timer(struct lpfc_hba * phba) **/ void lpfc_poll_timeout(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, fcp_poll_timer); + struct lpfc_hba *phba = timer_container_of(phba, t, fcp_poll_timer); if (phba->cfg_poll & ENABLE_FCP_RING_POLLING) { lpfc_sli_handle_fast_ring_event(phba, diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 2ebb073e4ef3..47bbcb78fb4d 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -3925,7 +3925,7 @@ void lpfc_poll_eratt(struct timer_list *t) uint32_t eratt = 0; uint64_t sli_intr, cnt; - phba = from_timer(phba, t, eratt_poll); + phba = timer_container_of(phba, t, eratt_poll); if (test_bit(FC_UNLOADING, &phba->pport->load_flag)) return; @@ -9125,7 +9125,7 @@ out_free_mbox: void lpfc_mbox_timeout(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, sli.mbox_tmo); + struct lpfc_hba *phba = timer_container_of(phba, t, sli.mbox_tmo); unsigned long iflag; uint32_t tmo_posted; @@ -15661,7 +15661,7 @@ lpfc_sli4_intr_handler(int irq, void *dev_id) void lpfc_sli4_poll_hbtimer(struct timer_list *t) { - struct lpfc_hba *phba = from_timer(phba, t, cpuhp_poll_timer); + struct lpfc_hba *phba = timer_container_of(phba, t, cpuhp_poll_timer); struct lpfc_queue *eq; rcu_read_lock(); diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c index b75f46c30759..b610cad83321 100644 --- a/drivers/scsi/megaraid/megaraid_mbox.c +++ b/drivers/scsi/megaraid/megaraid_mbox.c @@ -3836,7 +3836,7 @@ megaraid_sysfs_get_ldmap_done(uioc_t *uioc) static void megaraid_sysfs_get_ldmap_timeout(struct timer_list *t) { - struct uioc_timeout *timeout = from_timer(timeout, t, timer); + struct uioc_timeout *timeout = timer_container_of(timeout, t, timer); uioc_t *uioc = timeout->uioc; adapter_t *adapter = (adapter_t *)uioc->buf_vaddr; mraid_device_t *raid_dev = ADAP2RAIDDEV(adapter); diff --git a/drivers/scsi/megaraid/megaraid_mm.c b/drivers/scsi/megaraid/megaraid_mm.c index fd7fa7640a5e..87184e2538b0 100644 --- a/drivers/scsi/megaraid/megaraid_mm.c +++ b/drivers/scsi/megaraid/megaraid_mm.c @@ -783,7 +783,7 @@ ioctl_done(uioc_t *kioc) static void lld_timedout(struct timer_list *t) { - struct uioc_timeout *timeout = from_timer(timeout, t, timer); + struct uioc_timeout *timeout = timer_container_of(timeout, t, timer); uioc_t *kioc = timeout->uioc; kioc->status = -ETIME; diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index 5e33d411fa3d..3aac0e17cb00 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -2724,7 +2724,7 @@ out: static void megasas_sriov_heartbeat_handler(struct timer_list *t) { struct megasas_instance *instance = - from_timer(instance, t, sriov_heartbeat_timer); + timer_container_of(instance, t, sriov_heartbeat_timer); if (instance->hb_host_mem->HB.fwCounter != instance->hb_host_mem->HB.driverCounter) { diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c index 52ac10226cb0..6c46654b9cd9 100644 --- a/drivers/scsi/mvsas/mv_sas.c +++ b/drivers/scsi/mvsas/mv_sas.c @@ -1755,7 +1755,7 @@ static int mvs_handle_event(struct mvs_info *mvi, void *data, int handler) static void mvs_sig_time_out(struct timer_list *t) { - struct mvs_phy *phy = from_timer(phy, t, timer); + struct mvs_phy *phy = timer_container_of(phy, t, timer); struct mvs_info *mvi = phy->mvi; u8 phy_no; diff --git a/drivers/scsi/ncr53c8xx.c b/drivers/scsi/ncr53c8xx.c index 14ac81ec0aa0..34ba9b137789 100644 --- a/drivers/scsi/ncr53c8xx.c +++ b/drivers/scsi/ncr53c8xx.c @@ -7923,7 +7923,7 @@ irqreturn_t ncr53c8xx_intr(int irq, void *dev_id) static void ncr53c8xx_timeout(struct timer_list *t) { - struct ncb *np = from_timer(np, t, timer); + struct ncb *np = timer_container_of(np, t, timer); unsigned long flags; struct scsi_cmnd *done_list; diff --git a/drivers/scsi/pmcraid.c b/drivers/scsi/pmcraid.c index e0aeb206df8d..33f403e307eb 100644 --- a/drivers/scsi/pmcraid.c +++ b/drivers/scsi/pmcraid.c @@ -544,7 +544,7 @@ static void pmcraid_ioa_reset(struct pmcraid_cmd *); */ static void pmcraid_bist_done(struct timer_list *t) { - struct pmcraid_cmd *cmd = from_timer(cmd, t, timer); + struct pmcraid_cmd *cmd = timer_container_of(cmd, t, timer); struct pmcraid_instance *pinstance = cmd->drv_inst; unsigned long lock_flags; int rc; @@ -601,7 +601,7 @@ static void pmcraid_start_bist(struct pmcraid_cmd *cmd) */ static void pmcraid_reset_alert_done(struct timer_list *t) { - struct pmcraid_cmd *cmd = from_timer(cmd, t, timer); + struct pmcraid_cmd *cmd = timer_container_of(cmd, t, timer); struct pmcraid_instance *pinstance = cmd->drv_inst; u32 status = ioread32(pinstance->ioa_status); unsigned long lock_flags; @@ -685,7 +685,7 @@ static void pmcraid_reset_alert(struct pmcraid_cmd *cmd) */ static void pmcraid_timeout_handler(struct timer_list *t) { - struct pmcraid_cmd *cmd = from_timer(cmd, t, timer); + struct pmcraid_cmd *cmd = timer_container_of(cmd, t, timer); struct pmcraid_instance *pinstance = cmd->drv_inst; unsigned long lock_flags; diff --git a/drivers/scsi/qla1280.c b/drivers/scsi/qla1280.c index 078a9c80bce2..6af018f1ca22 100644 --- a/drivers/scsi/qla1280.c +++ b/drivers/scsi/qla1280.c @@ -721,7 +721,7 @@ enum action { static void qla1280_mailbox_timeout(struct timer_list *t) { - struct scsi_qla_host *ha = from_timer(ha, t, mailbox_timer); + struct scsi_qla_host *ha = timer_container_of(ha, t, mailbox_timer); struct device_reg __iomem *reg; reg = ha->iobase; diff --git a/drivers/scsi/qla2xxx/qla_edif.c b/drivers/scsi/qla2xxx/qla_edif.c index dcde55c8ee5d..91bbd3b75bff 100644 --- a/drivers/scsi/qla2xxx/qla_edif.c +++ b/drivers/scsi/qla2xxx/qla_edif.c @@ -94,7 +94,8 @@ static struct edif_list_entry *qla_edif_list_find_sa_index(fc_port_t *fcport, /* timeout called when no traffic and delayed rx sa_index delete */ static void qla2x00_sa_replace_iocb_timeout(struct timer_list *t) { - struct edif_list_entry *edif_entry = from_timer(edif_entry, t, timer); + struct edif_list_entry *edif_entry = timer_container_of(edif_entry, t, + timer); fc_port_t *fcport = edif_entry->fcport; struct scsi_qla_host *vha = fcport->vha; struct edif_sa_ctl *sa_ctl; diff --git a/drivers/scsi/qla2xxx/qla_init.c b/drivers/scsi/qla2xxx/qla_init.c index 0c2dd782b675..514934dd6f80 100644 --- a/drivers/scsi/qla2xxx/qla_init.c +++ b/drivers/scsi/qla2xxx/qla_init.c @@ -45,7 +45,7 @@ static void __qla24xx_handle_gpdb_event(scsi_qla_host_t *, struct event_arg *); void qla2x00_sp_timeout(struct timer_list *t) { - srb_t *sp = from_timer(sp, t, u.iocb_cmd.timer); + srb_t *sp = timer_container_of(sp, t, u.iocb_cmd.timer); struct srb_iocb *iocb; scsi_qla_host_t *vha = sp->vha; diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c index 288ce04fc2b1..d4b484c0fd9d 100644 --- a/drivers/scsi/qla2xxx/qla_os.c +++ b/drivers/scsi/qla2xxx/qla_os.c @@ -7380,7 +7380,7 @@ static void qla_wind_down_chip(scsi_qla_host_t *vha) void qla2x00_timer(struct timer_list *t) { - scsi_qla_host_t *vha = from_timer(vha, t, timer); + scsi_qla_host_t *vha = timer_container_of(vha, t, timer); unsigned long cpu_flags = 0; int start_dpc = 0; int index; diff --git a/drivers/scsi/qla4xxx/ql4_os.c b/drivers/scsi/qla4xxx/ql4_os.c index d540d66e6ffc..d4141656b204 100644 --- a/drivers/scsi/qla4xxx/ql4_os.c +++ b/drivers/scsi/qla4xxx/ql4_os.c @@ -4551,7 +4551,7 @@ static void qla4xxx_check_relogin_flash_ddb(struct iscsi_cls_session *cls_sess) **/ static void qla4xxx_timer(struct timer_list *t) { - struct scsi_qla_host *ha = from_timer(ha, t, timer); + struct scsi_qla_host *ha = timer_container_of(ha, t, timer); int start_dpc = 0; uint16_t w; diff --git a/drivers/scsi/scsi_devinfo.c b/drivers/scsi/scsi_devinfo.c index a348df895dca..8ff679e67bbf 100644 --- a/drivers/scsi/scsi_devinfo.c +++ b/drivers/scsi/scsi_devinfo.c @@ -836,7 +836,7 @@ int __init scsi_init_devinfo(void) goto out; for (i = 0; scsi_static_device_list[i].vendor; i++) { - error = scsi_dev_info_list_add(1 /* compatibile */, + error = scsi_dev_info_list_add(1 /* compatible */, scsi_static_device_list[i].vendor, scsi_static_device_list[i].model, NULL, diff --git a/drivers/scsi/smartpqi/smartpqi_init.c b/drivers/scsi/smartpqi/smartpqi_init.c index 1d784ee7671c..3d40a63e378d 100644 --- a/drivers/scsi/smartpqi/smartpqi_init.c +++ b/drivers/scsi/smartpqi/smartpqi_init.c @@ -3825,7 +3825,8 @@ static void pqi_heartbeat_timer_handler(struct timer_list *t) { int num_interrupts; u32 heartbeat_count; - struct pqi_ctrl_info *ctrl_info = from_timer(ctrl_info, t, heartbeat_timer); + struct pqi_ctrl_info *ctrl_info = timer_container_of(ctrl_info, t, + heartbeat_timer); pqi_check_ctrl_health(ctrl_info); if (pqi_ctrl_offline(ctrl_info)) diff --git a/drivers/scsi/sym53c8xx_2/sym_glue.c b/drivers/scsi/sym53c8xx_2/sym_glue.c index 1a6eb72ca281..57637a81776d 100644 --- a/drivers/scsi/sym53c8xx_2/sym_glue.c +++ b/drivers/scsi/sym53c8xx_2/sym_glue.c @@ -545,7 +545,7 @@ static irqreturn_t sym53c8xx_intr(int irq, void *dev_id) */ static void sym53c8xx_timer(struct timer_list *t) { - struct sym_hcb *np = from_timer(np, t, s.timer); + struct sym_hcb *np = timer_container_of(np, t, s.timer); unsigned long flags; spin_lock_irqsave(np->s.host->host_lock, flags); diff --git a/drivers/spi/spi-loongson-core.c b/drivers/spi/spi-loongson-core.c index 4fec226456d1..b46f072a0387 100644 --- a/drivers/spi/spi-loongson-core.c +++ b/drivers/spi/spi-loongson-core.c @@ -5,6 +5,7 @@ #include <linux/clk.h> #include <linux/delay.h> #include <linux/err.h> +#include <linux/export.h> #include <linux/init.h> #include <linux/interrupt.h> #include <linux/io.h> diff --git a/drivers/spi/spi-offload.c b/drivers/spi/spi-offload.c index e674097bf3be..d336f4d228d5 100644 --- a/drivers/spi/spi-offload.c +++ b/drivers/spi/spi-offload.c @@ -297,7 +297,7 @@ int spi_offload_trigger_enable(struct spi_offload *offload, if (trigger->ops->enable) { ret = trigger->ops->enable(trigger, config); if (ret) { - if (offload->ops->trigger_disable) + if (offload->ops && offload->ops->trigger_disable) offload->ops->trigger_disable(offload); return ret; } diff --git a/drivers/spi/spi-omap2-mcspi.c b/drivers/spi/spi-omap2-mcspi.c index 29c616e2c408..70bb74b3bd9c 100644 --- a/drivers/spi/spi-omap2-mcspi.c +++ b/drivers/spi/spi-omap2-mcspi.c @@ -134,6 +134,7 @@ struct omap2_mcspi { size_t max_xfer_len; u32 ref_clk_hz; bool use_multi_mode; + bool last_msg_kept_cs; }; struct omap2_mcspi_cs { @@ -1269,6 +1270,10 @@ static int omap2_mcspi_prepare_message(struct spi_controller *ctlr, * multi-mode is applicable. */ mcspi->use_multi_mode = true; + + if (mcspi->last_msg_kept_cs) + mcspi->use_multi_mode = false; + list_for_each_entry(tr, &msg->transfers, transfer_list) { if (!tr->bits_per_word) bits_per_word = msg->spi->bits_per_word; @@ -1287,18 +1292,19 @@ static int omap2_mcspi_prepare_message(struct spi_controller *ctlr, mcspi->use_multi_mode = false; } - /* Check if transfer asks to change the CS status after the transfer */ - if (!tr->cs_change) - mcspi->use_multi_mode = false; - - /* - * If at least one message is not compatible, switch back to single mode - * - * The bits_per_word of certain transfer can be different, but it will have no - * impact on the signal itself. - */ - if (!mcspi->use_multi_mode) - break; + if (list_is_last(&tr->transfer_list, &msg->transfers)) { + /* Check if transfer asks to keep the CS status after the whole message */ + if (tr->cs_change) { + mcspi->use_multi_mode = false; + mcspi->last_msg_kept_cs = true; + } else { + mcspi->last_msg_kept_cs = false; + } + } else { + /* Check if transfer asks to change the CS status after the transfer */ + if (!tr->cs_change) + mcspi->use_multi_mode = false; + } } omap2_mcspi_set_mode(ctlr); diff --git a/drivers/spi/spi-pci1xxxx.c b/drivers/spi/spi-pci1xxxx.c index c6489e90b8b9..e27642c4dea4 100644 --- a/drivers/spi/spi-pci1xxxx.c +++ b/drivers/spi/spi-pci1xxxx.c @@ -762,10 +762,10 @@ static int pci1xxxx_spi_probe(struct pci_dev *pdev, const struct pci_device_id * return -EINVAL; num_vector = pci_alloc_irq_vectors(pdev, 1, hw_inst_cnt, - PCI_IRQ_ALL_TYPES); + PCI_IRQ_INTX | PCI_IRQ_MSI); if (num_vector < 0) { dev_err(&pdev->dev, "Error allocating MSI vectors\n"); - return ret; + return num_vector; } init_completion(&spi_sub_ptr->spi_xfer_done); diff --git a/drivers/spi/spi-stm32-ospi.c b/drivers/spi/spi-stm32-ospi.c index 7c1fa55fbc47..4ab7e86f4bd5 100644 --- a/drivers/spi/spi-stm32-ospi.c +++ b/drivers/spi/spi-stm32-ospi.c @@ -804,7 +804,7 @@ static int stm32_ospi_get_resources(struct platform_device *pdev) return ret; } - ospi->rstc = devm_reset_control_array_get_exclusive(dev); + ospi->rstc = devm_reset_control_array_get_exclusive_released(dev); if (IS_ERR(ospi->rstc)) return dev_err_probe(dev, PTR_ERR(ospi->rstc), "Can't get reset\n"); @@ -936,12 +936,16 @@ static int stm32_ospi_probe(struct platform_device *pdev) if (ret < 0) goto err_pm_enable; - if (ospi->rstc) { - reset_control_assert(ospi->rstc); - udelay(2); - reset_control_deassert(ospi->rstc); + ret = reset_control_acquire(ospi->rstc); + if (ret) { + dev_err_probe(dev, ret, "Can not acquire reset %d\n", ret); + goto err_pm_resume; } + reset_control_assert(ospi->rstc); + udelay(2); + reset_control_deassert(ospi->rstc); + ret = spi_register_controller(ctrl); if (ret) { /* Disable ospi */ @@ -987,6 +991,8 @@ static void stm32_ospi_remove(struct platform_device *pdev) if (ospi->dma_chrx) dma_release_channel(ospi->dma_chrx); + reset_control_release(ospi->rstc); + pm_runtime_put_sync_suspend(ospi->dev); pm_runtime_force_suspend(ospi->dev); } @@ -997,6 +1003,8 @@ static int __maybe_unused stm32_ospi_suspend(struct device *dev) pinctrl_pm_select_sleep_state(dev); + reset_control_release(ospi->rstc); + return pm_runtime_force_suspend(ospi->dev); } @@ -1016,6 +1024,12 @@ static int __maybe_unused stm32_ospi_resume(struct device *dev) if (ret < 0) return ret; + ret = reset_control_acquire(ospi->rstc); + if (ret) { + dev_err(dev, "Can not acquire reset\n"); + return ret; + } + writel_relaxed(ospi->cr_reg, regs_base + OSPI_CR); writel_relaxed(ospi->dcr_reg, regs_base + OSPI_DCR1); pm_runtime_mark_last_busy(ospi->dev); diff --git a/drivers/staging/gpib/agilent_82357a/agilent_82357a.c b/drivers/staging/gpib/agilent_82357a/agilent_82357a.c index 454d46b8b677..b923dc606d1d 100644 --- a/drivers/staging/gpib/agilent_82357a/agilent_82357a.c +++ b/drivers/staging/gpib/agilent_82357a/agilent_82357a.c @@ -39,7 +39,8 @@ static void agilent_82357a_bulk_complete(struct urb *urb) static void agilent_82357a_timeout_handler(struct timer_list *t) { - struct agilent_82357a_priv *a_priv = from_timer(a_priv, t, bulk_timer); + struct agilent_82357a_priv *a_priv = timer_container_of(a_priv, t, + bulk_timer); struct agilent_82357a_urb_ctx *context = &a_priv->context; context->timed_out = 1; diff --git a/drivers/staging/gpib/common/gpib_os.c b/drivers/staging/gpib/common/gpib_os.c index 0678829ad14f..a193d64db033 100644 --- a/drivers/staging/gpib/common/gpib_os.c +++ b/drivers/staging/gpib/common/gpib_os.c @@ -85,7 +85,7 @@ static int pop_gpib_event_nolock(struct gpib_board *board, static void watchdog_timeout(struct timer_list *t) { - struct gpib_board *board = from_timer(board, t, timer); + struct gpib_board *board = timer_container_of(board, t, timer); set_bit(TIMO_NUM, &board->status); wake_up_interruptible(&board->wait); @@ -133,7 +133,8 @@ static inline int pseudo_irq_period(void) static void pseudo_irq_handler(struct timer_list *t) { - struct gpib_pseudo_irq *pseudo_irq = from_timer(pseudo_irq, t, timer); + struct gpib_pseudo_irq *pseudo_irq = timer_container_of(pseudo_irq, t, + timer); if (pseudo_irq->handler) pseudo_irq->handler(0, pseudo_irq->board); diff --git a/drivers/staging/gpib/common/iblib.c b/drivers/staging/gpib/common/iblib.c index 512bd75900ec..549280d9a6e9 100644 --- a/drivers/staging/gpib/common/iblib.c +++ b/drivers/staging/gpib/common/iblib.c @@ -574,7 +574,7 @@ struct wait_info { static void wait_timeout(struct timer_list *t) { - struct wait_info *winfo = from_timer(winfo, t, timer); + struct wait_info *winfo = timer_container_of(winfo, t, timer); winfo->timed_out = 1; wake_up_interruptible(&winfo->board->wait); diff --git a/drivers/staging/gpib/ni_usb/ni_usb_gpib.c b/drivers/staging/gpib/ni_usb/ni_usb_gpib.c index 9ec850c4749f..7cf25c95787f 100644 --- a/drivers/staging/gpib/ni_usb/ni_usb_gpib.c +++ b/drivers/staging/gpib/ni_usb/ni_usb_gpib.c @@ -93,7 +93,8 @@ static void ni_usb_bulk_complete(struct urb *urb) static void ni_usb_timeout_handler(struct timer_list *t) { - struct ni_usb_priv *ni_priv = from_timer(ni_priv, t, bulk_timer); + struct ni_usb_priv *ni_priv = timer_container_of(ni_priv, t, + bulk_timer); struct ni_usb_urb_ctx *context = &ni_priv->context; context->timed_out = 1; diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c index 2855ba2296ac..77360bfe081a 100644 --- a/drivers/staging/media/imx/imx-ic-prpencvf.c +++ b/drivers/staging/media/imx/imx-ic-prpencvf.c @@ -295,7 +295,7 @@ static irqreturn_t prp_nfb4eof_interrupt(int irq, void *dev_id) */ static void prp_eof_timeout(struct timer_list *t) { - struct prp_priv *priv = from_timer(priv, t, eof_timeout_timer); + struct prp_priv *priv = timer_container_of(priv, t, eof_timeout_timer); struct imx_media_video_dev *vdev = priv->vdev; struct imx_ic_priv *ic_priv = priv->ic_priv; diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c index f1d7fce8c020..a7cd3ef95fc3 100644 --- a/drivers/staging/media/imx/imx-media-csi.c +++ b/drivers/staging/media/imx/imx-media-csi.c @@ -356,7 +356,7 @@ static irqreturn_t csi_idmac_nfb4eof_interrupt(int irq, void *dev_id) */ static void csi_idmac_eof_timeout(struct timer_list *t) { - struct csi_priv *priv = from_timer(priv, t, eof_timeout_timer); + struct csi_priv *priv = timer_container_of(priv, t, eof_timeout_timer); struct imx_media_video_dev *vdev = priv->vdev; v4l2_err(&priv->sd, "EOF timeout\n"); diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme.c b/drivers/staging/rtl8723bs/core/rtw_mlme.c index 1d23ea7d6f59..6301dbbcc472 100644 --- a/drivers/staging/rtl8723bs/core/rtw_mlme.c +++ b/drivers/staging/rtl8723bs/core/rtw_mlme.c @@ -1460,8 +1460,8 @@ void rtw_wmm_event_callback(struct adapter *padapter, u8 *pbuf) */ void _rtw_join_timeout_handler(struct timer_list *t) { - struct adapter *adapter = from_timer(adapter, t, - mlmepriv.assoc_timer); + struct adapter *adapter = timer_container_of(adapter, t, + mlmepriv.assoc_timer); struct mlme_priv *pmlmepriv = &adapter->mlmepriv; if (adapter->bDriverStopped || adapter->bSurpriseRemoved) @@ -1504,8 +1504,8 @@ void _rtw_join_timeout_handler(struct timer_list *t) */ void rtw_scan_timeout_handler(struct timer_list *t) { - struct adapter *adapter = from_timer(adapter, t, - mlmepriv.scan_to_timer); + struct adapter *adapter = timer_container_of(adapter, t, + mlmepriv.scan_to_timer); struct mlme_priv *pmlmepriv = &adapter->mlmepriv; spin_lock_bh(&pmlmepriv->lock); diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c index e74fb7d5dc37..bc980d21d50e 100644 --- a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c +++ b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c @@ -5088,7 +5088,7 @@ void linked_status_chk(struct adapter *padapter) void survey_timer_hdl(struct timer_list *t) { struct adapter *padapter = - from_timer(padapter, t, mlmeextpriv.survey_timer); + timer_container_of(padapter, t, mlmeextpriv.survey_timer); struct cmd_obj *ph2c; struct sitesurvey_parm *psurveyPara; struct cmd_priv *pcmdpriv = &padapter->cmdpriv; @@ -5123,7 +5123,7 @@ void survey_timer_hdl(struct timer_list *t) void link_timer_hdl(struct timer_list *t) { struct adapter *padapter = - from_timer(padapter, t, mlmeextpriv.link_timer); + timer_container_of(padapter, t, mlmeextpriv.link_timer); /* static unsigned int rx_pkt = 0; */ /* static u64 tx_cnt = 0; */ /* struct xmit_priv *pxmitpriv = &(padapter->xmitpriv); */ @@ -5161,7 +5161,7 @@ void link_timer_hdl(struct timer_list *t) void addba_timer_hdl(struct timer_list *t) { - struct sta_info *psta = from_timer(psta, t, addba_retry_timer); + struct sta_info *psta = timer_container_of(psta, t, addba_retry_timer); struct ht_priv *phtpriv; if (!psta) @@ -5179,7 +5179,7 @@ void addba_timer_hdl(struct timer_list *t) void sa_query_timer_hdl(struct timer_list *t) { struct adapter *padapter = - from_timer(padapter, t, mlmeextpriv.sa_query_timer); + timer_container_of(padapter, t, mlmeextpriv.sa_query_timer); struct mlme_priv *pmlmepriv = &padapter->mlmepriv; /* disconnect */ spin_lock_bh(&pmlmepriv->lock); diff --git a/drivers/staging/rtl8723bs/core/rtw_pwrctrl.c b/drivers/staging/rtl8723bs/core/rtw_pwrctrl.c index 44f7c19308a5..6a2583d0d3eb 100644 --- a/drivers/staging/rtl8723bs/core/rtw_pwrctrl.c +++ b/drivers/staging/rtl8723bs/core/rtw_pwrctrl.c @@ -175,7 +175,7 @@ exit: static void pwr_state_check_handler(struct timer_list *t) { struct pwrctrl_priv *pwrctrlpriv = - from_timer(pwrctrlpriv, t, pwr_state_check_timer); + timer_container_of(pwrctrlpriv, t, pwr_state_check_timer); struct adapter *padapter = pwrctrlpriv->adapter; rtw_ps_cmd(padapter); @@ -674,7 +674,8 @@ exit: */ static void pwr_rpwm_timeout_handler(struct timer_list *t) { - struct pwrctrl_priv *pwrpriv = from_timer(pwrpriv, t, pwr_rpwm_timer); + struct pwrctrl_priv *pwrpriv = timer_container_of(pwrpriv, t, + pwr_rpwm_timer); if ((pwrpriv->rpwm == pwrpriv->cpwm) || (pwrpriv->cpwm >= PS_STATE_S2)) return; diff --git a/drivers/staging/rtl8723bs/core/rtw_recv.c b/drivers/staging/rtl8723bs/core/rtw_recv.c index 709a606be7c9..8ae527b6e0d6 100644 --- a/drivers/staging/rtl8723bs/core/rtw_recv.c +++ b/drivers/staging/rtl8723bs/core/rtw_recv.c @@ -1908,7 +1908,7 @@ _err_exit: void rtw_reordering_ctrl_timeout_handler(struct timer_list *t) { struct recv_reorder_ctrl *preorder_ctrl = - from_timer(preorder_ctrl, t, reordering_ctrl_timer); + timer_container_of(preorder_ctrl, t, reordering_ctrl_timer); struct adapter *padapter = preorder_ctrl->padapter; struct __queue *ppending_recvframe_queue = &preorder_ctrl->pending_recvframe_queue; @@ -2087,7 +2087,7 @@ _recv_entry_drop: static void rtw_signal_stat_timer_hdl(struct timer_list *t) { struct adapter *adapter = - from_timer(adapter, t, recvpriv.signal_stat_timer); + timer_container_of(adapter, t, recvpriv.signal_stat_timer); struct recv_priv *recvpriv = &adapter->recvpriv; u32 tmp_s, tmp_q; diff --git a/drivers/staging/rtl8723bs/os_dep/mlme_linux.c b/drivers/staging/rtl8723bs/os_dep/mlme_linux.c index 1904e82a24b5..fd4ae870a617 100644 --- a/drivers/staging/rtl8723bs/os_dep/mlme_linux.c +++ b/drivers/staging/rtl8723bs/os_dep/mlme_linux.c @@ -9,7 +9,7 @@ static void _dynamic_check_timer_handler(struct timer_list *t) { struct adapter *adapter = - from_timer(adapter, t, mlmepriv.dynamic_chk_timer); + timer_container_of(adapter, t, mlmepriv.dynamic_chk_timer); rtw_dynamic_check_timer_handler(adapter); @@ -19,7 +19,7 @@ static void _dynamic_check_timer_handler(struct timer_list *t) static void _rtw_set_scan_deny_timer_hdl(struct timer_list *t) { struct adapter *adapter = - from_timer(adapter, t, mlmepriv.set_scan_deny_timer); + timer_container_of(adapter, t, mlmepriv.set_scan_deny_timer); rtw_clear_scan_deny(adapter); } diff --git a/drivers/target/iscsi/iscsi_target_erl0.c b/drivers/target/iscsi/iscsi_target_erl0.c index f0d7eebfcad6..24db6b07493e 100644 --- a/drivers/target/iscsi/iscsi_target_erl0.c +++ b/drivers/target/iscsi/iscsi_target_erl0.c @@ -743,7 +743,8 @@ int iscsit_check_post_dataout( void iscsit_handle_time2retain_timeout(struct timer_list *t) { - struct iscsit_session *sess = from_timer(sess, t, time2retain_timer); + struct iscsit_session *sess = timer_container_of(sess, t, + time2retain_timer); struct iscsi_portal_group *tpg = sess->tpg; struct se_portal_group *se_tpg = &tpg->tpg_se_tpg; diff --git a/drivers/target/iscsi/iscsi_target_erl1.c b/drivers/target/iscsi/iscsi_target_erl1.c index e7c3c4cdaae4..d8ca06e697d6 100644 --- a/drivers/target/iscsi/iscsi_target_erl1.c +++ b/drivers/target/iscsi/iscsi_target_erl1.c @@ -1094,7 +1094,7 @@ void iscsit_handle_dataout_timeout(struct timer_list *t) { u32 pdu_length = 0, pdu_offset = 0; u32 r2t_length = 0, r2t_offset = 0; - struct iscsit_cmd *cmd = from_timer(cmd, t, dataout_timer); + struct iscsit_cmd *cmd = timer_container_of(cmd, t, dataout_timer); struct iscsit_conn *conn = cmd->conn; struct iscsit_session *sess = NULL; struct iscsi_node_attrib *na; diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c index 0bd62ab9a1cd..5e6cf34929b5 100644 --- a/drivers/target/iscsi/iscsi_target_util.c +++ b/drivers/target/iscsi/iscsi_target_util.c @@ -851,7 +851,8 @@ static int iscsit_add_nopin(struct iscsit_conn *conn, int want_response) void iscsit_handle_nopin_response_timeout(struct timer_list *t) { - struct iscsit_conn *conn = from_timer(conn, t, nopin_response_timer); + struct iscsit_conn *conn = timer_container_of(conn, t, + nopin_response_timer); struct iscsit_session *sess = conn->sess; iscsit_inc_conn_usage_count(conn); @@ -931,7 +932,7 @@ void iscsit_stop_nopin_response_timer(struct iscsit_conn *conn) void iscsit_handle_nopin_timeout(struct timer_list *t) { - struct iscsit_conn *conn = from_timer(conn, t, nopin_timer); + struct iscsit_conn *conn = timer_container_of(conn, t, nopin_timer); iscsit_inc_conn_usage_count(conn); @@ -998,7 +999,7 @@ void iscsit_stop_nopin_timer(struct iscsit_conn *conn) void iscsit_login_timeout(struct timer_list *t) { - struct iscsit_conn *conn = from_timer(conn, t, login_timer); + struct iscsit_conn *conn = timer_container_of(conn, t, login_timer); struct iscsi_login *login = conn->login; pr_debug("Entering iscsi_target_login_timeout >>>>>>>>>>>>>>>>>>>\n"); diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index 43872ccc07cc..3fd963612775 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -1564,7 +1564,7 @@ static void tcmu_device_timedout(struct tcmu_dev *udev) static void tcmu_cmd_timedout(struct timer_list *t) { - struct tcmu_dev *udev = from_timer(udev, t, cmd_timer); + struct tcmu_dev *udev = timer_container_of(udev, t, cmd_timer); pr_debug("%s cmd timeout has expired\n", udev->name); tcmu_device_timedout(udev); @@ -1572,7 +1572,7 @@ static void tcmu_cmd_timedout(struct timer_list *t) static void tcmu_qfull_timedout(struct timer_list *t) { - struct tcmu_dev *udev = from_timer(udev, t, qfull_timer); + struct tcmu_dev *udev = timer_container_of(udev, t, qfull_timer); pr_debug("%s qfull timeout has expired\n", udev->name); tcmu_device_timedout(udev); diff --git a/drivers/tty/ipwireless/hardware.c b/drivers/tty/ipwireless/hardware.c index c13f52337035..e18848267be4 100644 --- a/drivers/tty/ipwireless/hardware.c +++ b/drivers/tty/ipwireless/hardware.c @@ -1676,7 +1676,7 @@ void ipwireless_init_hardware_v2_v3(struct ipw_hardware *hw) static void ipwireless_setup_timer(struct timer_list *t) { - struct ipw_hardware *hw = from_timer(hw, t, setup_timer); + struct ipw_hardware *hw = timer_container_of(hw, t, setup_timer); hw->init_loops++; diff --git a/drivers/tty/mips_ejtag_fdc.c b/drivers/tty/mips_ejtag_fdc.c index fa47bfcf9e86..ba7690c3e15e 100644 --- a/drivers/tty/mips_ejtag_fdc.c +++ b/drivers/tty/mips_ejtag_fdc.c @@ -683,7 +683,8 @@ static irqreturn_t mips_ejtag_fdc_isr(int irq, void *dev_id) */ static void mips_ejtag_fdc_tty_timer(struct timer_list *t) { - struct mips_ejtag_fdc_tty *priv = from_timer(priv, t, poll_timer); + struct mips_ejtag_fdc_tty *priv = timer_container_of(priv, t, + poll_timer); mips_ejtag_fdc_handle(priv); if (!priv->removing) diff --git a/drivers/tty/n_gsm.c b/drivers/tty/n_gsm.c index 40a336ef8c7e..7fc535452c0b 100644 --- a/drivers/tty/n_gsm.c +++ b/drivers/tty/n_gsm.c @@ -1971,7 +1971,7 @@ static void gsm_control_response(struct gsm_mux *gsm, unsigned int command, static void gsm_control_keep_alive(struct timer_list *t) { - struct gsm_mux *gsm = from_timer(gsm, t, ka_timer); + struct gsm_mux *gsm = timer_container_of(gsm, t, ka_timer); unsigned long flags; spin_lock_irqsave(&gsm->control_lock, flags); @@ -2028,7 +2028,7 @@ static void gsm_control_transmit(struct gsm_mux *gsm, struct gsm_control *ctrl) static void gsm_control_retransmit(struct timer_list *t) { - struct gsm_mux *gsm = from_timer(gsm, t, t2_timer); + struct gsm_mux *gsm = timer_container_of(gsm, t, t2_timer); struct gsm_control *ctrl; unsigned long flags; spin_lock_irqsave(&gsm->control_lock, flags); @@ -2229,7 +2229,7 @@ static int gsm_dlci_negotiate(struct gsm_dlci *dlci) static void gsm_dlci_t1(struct timer_list *t) { - struct gsm_dlci *dlci = from_timer(dlci, t, t1); + struct gsm_dlci *dlci = timer_container_of(dlci, t, t1); struct gsm_mux *gsm = dlci->gsm; switch (dlci->state) { @@ -2489,7 +2489,7 @@ static void gsm_dlci_command(struct gsm_dlci *dlci, const u8 *data, int len) */ static void gsm_kick_timer(struct timer_list *t) { - struct gsm_mux *gsm = from_timer(gsm, t, kick_timer); + struct gsm_mux *gsm = timer_container_of(gsm, t, kick_timer); unsigned long flags; int sent = 0; diff --git a/drivers/tty/serial/8250/8250_aspeed_vuart.c b/drivers/tty/serial/8250/8250_aspeed_vuart.c index 392447038bfb..26fc0464f1cc 100644 --- a/drivers/tty/serial/8250/8250_aspeed_vuart.c +++ b/drivers/tty/serial/8250/8250_aspeed_vuart.c @@ -304,7 +304,8 @@ static void aspeed_vuart_unthrottle(struct uart_port *port) static void aspeed_vuart_unthrottle_exp(struct timer_list *timer) { - struct aspeed_vuart *vuart = from_timer(vuart, timer, unthrottle_timer); + struct aspeed_vuart *vuart = timer_container_of(vuart, timer, + unthrottle_timer); struct uart_8250_port *up = vuart->port; if (!tty_buffer_space_avail(&up->port.state->port)) { diff --git a/drivers/tty/serial/8250/8250_core.c b/drivers/tty/serial/8250/8250_core.c index 68994a964321..7a6050f1c094 100644 --- a/drivers/tty/serial/8250/8250_core.c +++ b/drivers/tty/serial/8250/8250_core.c @@ -206,7 +206,7 @@ static void serial_unlink_irq_chain(struct uart_8250_port *up) */ static void serial8250_timeout(struct timer_list *t) { - struct uart_8250_port *up = from_timer(up, t, timer); + struct uart_8250_port *up = timer_container_of(up, t, timer); up->port.handle_irq(&up->port); mod_timer(&up->timer, jiffies + uart_poll_timeout(&up->port)); @@ -214,7 +214,7 @@ static void serial8250_timeout(struct timer_list *t) static void serial8250_backup_timeout(struct timer_list *t) { - struct uart_8250_port *up = from_timer(up, t, timer); + struct uart_8250_port *up = timer_container_of(up, t, timer); unsigned int iir, ier = 0, lsr; unsigned long flags; diff --git a/drivers/tty/serial/altera_uart.c b/drivers/tty/serial/altera_uart.c index 011f38681131..837991dc4db9 100644 --- a/drivers/tty/serial/altera_uart.c +++ b/drivers/tty/serial/altera_uart.c @@ -275,7 +275,7 @@ static irqreturn_t altera_uart_interrupt(int irq, void *data) static void altera_uart_timer(struct timer_list *t) { - struct altera_uart *pp = from_timer(pp, t, tmr); + struct altera_uart *pp = timer_container_of(pp, t, tmr); struct uart_port *port = &pp->port; altera_uart_interrupt(0, port); diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c index 421ac22555df..22939841b1de 100644 --- a/drivers/tty/serial/amba-pl011.c +++ b/drivers/tty/serial/amba-pl011.c @@ -1046,7 +1046,7 @@ static inline void pl011_dma_rx_stop(struct uart_amba_port *uap) */ static void pl011_dma_rx_poll(struct timer_list *t) { - struct uart_amba_port *uap = from_timer(uap, t, dmarx.timer); + struct uart_amba_port *uap = timer_container_of(uap, t, dmarx.timer); struct tty_port *port = &uap->port.state->port; struct pl011_dmarx_data *dmarx = &uap->dmarx; struct dma_chan *rxchan = uap->dmarx.chan; diff --git a/drivers/tty/serial/atmel_serial.c b/drivers/tty/serial/atmel_serial.c index 18dba502144d..08dd8f887956 100644 --- a/drivers/tty/serial/atmel_serial.c +++ b/drivers/tty/serial/atmel_serial.c @@ -1266,8 +1266,8 @@ chan_err: static void atmel_uart_timer_callback(struct timer_list *t) { - struct atmel_uart_port *atmel_port = from_timer(atmel_port, t, - uart_timer); + struct atmel_uart_port *atmel_port = timer_container_of(atmel_port, t, + uart_timer); struct uart_port *port = &atmel_port->uart; if (!atomic_read(&atmel_port->tasklet_shutdown)) { diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c index dff6a6c57b5f..2790b4078e7e 100644 --- a/drivers/tty/serial/fsl_lpuart.c +++ b/drivers/tty/serial/fsl_lpuart.c @@ -1307,7 +1307,7 @@ static irqreturn_t lpuart32_int(int irq, void *dev_id) */ static void lpuart_timer_func(struct timer_list *t) { - struct lpuart_port *sport = from_timer(sport, t, lpuart_timer); + struct lpuart_port *sport = timer_container_of(sport, t, lpuart_timer); enum dma_status dmastat; struct dma_chan *chan = sport->dma_rx_chan; struct circ_buf *ring = &sport->rx_ring; diff --git a/drivers/tty/serial/imx.c b/drivers/tty/serial/imx.c index e4b6f1bfdc95..bd02ee898f5d 100644 --- a/drivers/tty/serial/imx.c +++ b/drivers/tty/serial/imx.c @@ -1165,7 +1165,7 @@ static void imx_uart_break_ctl(struct uart_port *port, int break_state) */ static void imx_uart_timeout(struct timer_list *t) { - struct imx_port *sport = from_timer(sport, t, timer); + struct imx_port *sport = timer_container_of(sport, t, timer); unsigned long flags; if (sport->port.state) { diff --git a/drivers/tty/serial/liteuart.c b/drivers/tty/serial/liteuart.c index 3a0960c97c77..6429e8f11f36 100644 --- a/drivers/tty/serial/liteuart.c +++ b/drivers/tty/serial/liteuart.c @@ -152,7 +152,7 @@ static irqreturn_t liteuart_interrupt(int irq, void *data) static void liteuart_timer(struct timer_list *t) { - struct liteuart_port *uart = from_timer(uart, t, timer); + struct liteuart_port *uart = timer_container_of(uart, t, timer); struct uart_port *port = &uart->port; liteuart_interrupt(0, port); diff --git a/drivers/tty/serial/max3100.c b/drivers/tty/serial/max3100.c index d28a2ebfa29f..67d80f8f801e 100644 --- a/drivers/tty/serial/max3100.c +++ b/drivers/tty/serial/max3100.c @@ -309,7 +309,7 @@ static void max3100_dowork(struct max3100_port *s) static void max3100_timeout(struct timer_list *t) { - struct max3100_port *s = from_timer(s, t, timer); + struct max3100_port *s = timer_container_of(s, t, timer); max3100_dowork(s); mod_timer(&s->timer, jiffies + uart_poll_timeout(&s->port)); diff --git a/drivers/tty/serial/sa1100.c b/drivers/tty/serial/sa1100.c index 8587ebbe1073..72b1bb76415c 100644 --- a/drivers/tty/serial/sa1100.c +++ b/drivers/tty/serial/sa1100.c @@ -111,7 +111,7 @@ static void sa1100_mctrl_check(struct sa1100_port *sport) */ static void sa1100_timeout(struct timer_list *t) { - struct sa1100_port *sport = from_timer(sport, t, timer); + struct sa1100_port *sport = timer_container_of(sport, t, timer); unsigned long flags; if (sport->port.state) { diff --git a/drivers/tty/serial/sccnxp.c b/drivers/tty/serial/sccnxp.c index 553e3c1321ca..4ceca11ce600 100644 --- a/drivers/tty/serial/sccnxp.c +++ b/drivers/tty/serial/sccnxp.c @@ -500,7 +500,7 @@ static void sccnxp_handle_events(struct sccnxp_port *s) static void sccnxp_timer(struct timer_list *t) { - struct sccnxp_port *s = from_timer(s, t, timer); + struct sccnxp_port *s = timer_container_of(s, t, timer); unsigned long flags; spin_lock_irqsave(&s->lock, flags); diff --git a/drivers/tty/serial/sh-sci.c b/drivers/tty/serial/sh-sci.c index ff1986dc6af3..1c356544a832 100644 --- a/drivers/tty/serial/sh-sci.c +++ b/drivers/tty/serial/sh-sci.c @@ -1120,7 +1120,7 @@ static int scif_rtrg_enabled(struct uart_port *port) static void rx_fifo_timer_fn(struct timer_list *t) { - struct sci_port *s = from_timer(s, t, rx_fifo_timer); + struct sci_port *s = timer_container_of(s, t, rx_fifo_timer); struct uart_port *port = &s->port; dev_dbg(port->dev, "Rx timed out\n"); diff --git a/drivers/tty/synclink_gt.c b/drivers/tty/synclink_gt.c index 4c703f42680d..3865b10d2d43 100644 --- a/drivers/tty/synclink_gt.c +++ b/drivers/tty/synclink_gt.c @@ -5002,7 +5002,7 @@ static int adapter_test(struct slgt_info *info) */ static void tx_timeout(struct timer_list *t) { - struct slgt_info *info = from_timer(info, t, tx_timer); + struct slgt_info *info = timer_container_of(info, t, tx_timer); unsigned long flags; DBGINFO(("%s tx_timeout\n", info->device_name)); @@ -5026,7 +5026,7 @@ static void tx_timeout(struct timer_list *t) */ static void rx_timeout(struct timer_list *t) { - struct slgt_info *info = from_timer(info, t, rx_timer); + struct slgt_info *info = timer_container_of(info, t, rx_timer); unsigned long flags; DBGINFO(("%s rx_timeout\n", info->device_name)); diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c index 6853c4660e7c..d77c03d22227 100644 --- a/drivers/tty/sysrq.c +++ b/drivers/tty/sysrq.c @@ -712,7 +712,8 @@ static void sysrq_parse_reset_sequence(struct sysrq_state *state) static void sysrq_do_reset(struct timer_list *t) { - struct sysrq_state *state = from_timer(state, t, keyreset_timer); + struct sysrq_state *state = timer_container_of(state, t, + keyreset_timer); state->reset_requested = true; diff --git a/drivers/tty/vcc.c b/drivers/tty/vcc.c index 7ac3048377d5..2960031ace72 100644 --- a/drivers/tty/vcc.c +++ b/drivers/tty/vcc.c @@ -356,7 +356,7 @@ done: static void vcc_rx_timer(struct timer_list *t) { - struct vcc_port *port = from_timer(port, t, rx_timer); + struct vcc_port *port = timer_container_of(port, t, rx_timer); struct vio_driver_state *vio; unsigned long flags; int rv; @@ -382,7 +382,7 @@ done: static void vcc_tx_timer(struct timer_list *t) { - struct vcc_port *port = from_timer(port, t, tx_timer); + struct vcc_port *port = timer_container_of(port, t, tx_timer); struct vio_vcc *pkt; unsigned long flags; size_t tosend = 0; diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 76cedd30c274..4410e7d93b7d 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -1397,6 +1397,7 @@ static int ufshcd_clock_scaling_prepare(struct ufs_hba *hba, u64 timeout_us) * make sure that there are no outstanding requests when * clock scaling is in progress */ + mutex_lock(&hba->host->scan_mutex); blk_mq_quiesce_tagset(&hba->host->tag_set); mutex_lock(&hba->wb_mutex); down_write(&hba->clk_scaling_lock); @@ -1407,6 +1408,7 @@ static int ufshcd_clock_scaling_prepare(struct ufs_hba *hba, u64 timeout_us) up_write(&hba->clk_scaling_lock); mutex_unlock(&hba->wb_mutex); blk_mq_unquiesce_tagset(&hba->host->tag_set); + mutex_unlock(&hba->host->scan_mutex); goto out; } @@ -1428,6 +1430,7 @@ static void ufshcd_clock_scaling_unprepare(struct ufs_hba *hba, int err) mutex_unlock(&hba->wb_mutex); blk_mq_unquiesce_tagset(&hba->host->tag_set); + mutex_unlock(&hba->host->scan_mutex); ufshcd_release(hba); } diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 37887ec68412..18a978452001 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -122,7 +122,9 @@ static const struct { }; static void ufs_qcom_get_default_testbus_cfg(struct ufs_qcom_host *host); -static int ufs_qcom_set_core_clk_ctrl(struct ufs_hba *hba, unsigned long freq); +static unsigned long ufs_qcom_opp_freq_to_clk_freq(struct ufs_hba *hba, + unsigned long freq, char *name); +static int ufs_qcom_set_core_clk_ctrl(struct ufs_hba *hba, bool is_scale_up, unsigned long freq); static struct ufs_qcom_host *rcdev_to_ufs_host(struct reset_controller_dev *rcd) { @@ -506,10 +508,9 @@ static int ufs_qcom_power_up_sequence(struct ufs_hba *hba) if (ret) return ret; - if (phy->power_count) { + if (phy->power_count) phy_power_off(phy); - phy_exit(phy); - } + /* phy initialization - calibrate the phy */ ret = phy_init(phy); @@ -597,13 +598,14 @@ static int ufs_qcom_hce_enable_notify(struct ufs_hba *hba, * * @hba: host controller instance * @is_pre_scale_up: flag to check if pre scale up condition. + * @freq: target opp freq * Return: zero for success and non-zero in case of a failure. */ -static int ufs_qcom_cfg_timers(struct ufs_hba *hba, bool is_pre_scale_up) +static int ufs_qcom_cfg_timers(struct ufs_hba *hba, bool is_pre_scale_up, unsigned long freq) { struct ufs_qcom_host *host = ufshcd_get_variant(hba); struct ufs_clk_info *clki; - unsigned long core_clk_rate = 0; + unsigned long clk_freq = 0; u32 core_clk_cycles_per_us; /* @@ -615,22 +617,34 @@ static int ufs_qcom_cfg_timers(struct ufs_hba *hba, bool is_pre_scale_up) if (host->hw_ver.major < 4 && !ufshcd_is_intr_aggr_allowed(hba)) return 0; + if (hba->use_pm_opp && freq != ULONG_MAX) { + clk_freq = ufs_qcom_opp_freq_to_clk_freq(hba, freq, "core_clk"); + if (clk_freq) + goto cfg_timers; + } + list_for_each_entry(clki, &hba->clk_list_head, list) { if (!strcmp(clki->name, "core_clk")) { + if (freq == ULONG_MAX) { + clk_freq = clki->max_freq; + break; + } + if (is_pre_scale_up) - core_clk_rate = clki->max_freq; + clk_freq = clki->max_freq; else - core_clk_rate = clk_get_rate(clki->clk); + clk_freq = clk_get_rate(clki->clk); break; } } +cfg_timers: /* If frequency is smaller than 1MHz, set to 1MHz */ - if (core_clk_rate < DEFAULT_CLK_RATE_HZ) - core_clk_rate = DEFAULT_CLK_RATE_HZ; + if (clk_freq < DEFAULT_CLK_RATE_HZ) + clk_freq = DEFAULT_CLK_RATE_HZ; - core_clk_cycles_per_us = core_clk_rate / USEC_PER_SEC; + core_clk_cycles_per_us = clk_freq / USEC_PER_SEC; if (ufshcd_readl(hba, REG_UFS_SYS1CLK_1US) != core_clk_cycles_per_us) { ufshcd_writel(hba, core_clk_cycles_per_us, REG_UFS_SYS1CLK_1US); /* @@ -650,13 +664,13 @@ static int ufs_qcom_link_startup_notify(struct ufs_hba *hba, switch (status) { case PRE_CHANGE: - if (ufs_qcom_cfg_timers(hba, false)) { + if (ufs_qcom_cfg_timers(hba, false, ULONG_MAX)) { dev_err(hba->dev, "%s: ufs_qcom_cfg_timers() failed\n", __func__); return -EINVAL; } - err = ufs_qcom_set_core_clk_ctrl(hba, ULONG_MAX); + err = ufs_qcom_set_core_clk_ctrl(hba, true, ULONG_MAX); if (err) dev_err(hba->dev, "cfg core clk ctrl failed\n"); /* @@ -928,17 +942,6 @@ static int ufs_qcom_pwr_change_notify(struct ufs_hba *hba, break; case POST_CHANGE: - if (ufs_qcom_cfg_timers(hba, false)) { - dev_err(hba->dev, "%s: ufs_qcom_cfg_timers() failed\n", - __func__); - /* - * we return error code at the end of the routine, - * but continue to configure UFS_PHY_TX_LANE_ENABLE - * and bus voting as usual - */ - ret = -EINVAL; - } - /* cache the power mode parameters to use internally */ memcpy(&host->dev_req_params, dev_req_params, sizeof(*dev_req_params)); @@ -1414,29 +1417,46 @@ static int ufs_qcom_set_clk_40ns_cycles(struct ufs_hba *hba, return ufshcd_dme_set(hba, UIC_ARG_MIB(PA_VS_CORE_CLK_40NS_CYCLES), reg); } -static int ufs_qcom_set_core_clk_ctrl(struct ufs_hba *hba, unsigned long freq) +static int ufs_qcom_set_core_clk_ctrl(struct ufs_hba *hba, bool is_scale_up, unsigned long freq) { struct ufs_qcom_host *host = ufshcd_get_variant(hba); struct list_head *head = &hba->clk_list_head; struct ufs_clk_info *clki; u32 cycles_in_1us = 0; u32 core_clk_ctrl_reg; + unsigned long clk_freq; int err; + if (hba->use_pm_opp && freq != ULONG_MAX) { + clk_freq = ufs_qcom_opp_freq_to_clk_freq(hba, freq, "core_clk_unipro"); + if (clk_freq) { + cycles_in_1us = ceil(clk_freq, HZ_PER_MHZ); + goto set_core_clk_ctrl; + } + } + list_for_each_entry(clki, head, list) { if (!IS_ERR_OR_NULL(clki->clk) && !strcmp(clki->name, "core_clk_unipro")) { - if (!clki->max_freq) + if (!clki->max_freq) { cycles_in_1us = 150; /* default for backwards compatibility */ - else if (freq == ULONG_MAX) + break; + } + + if (freq == ULONG_MAX) { cycles_in_1us = ceil(clki->max_freq, HZ_PER_MHZ); - else - cycles_in_1us = ceil(freq, HZ_PER_MHZ); + break; + } + if (is_scale_up) + cycles_in_1us = ceil(clki->max_freq, HZ_PER_MHZ); + else + cycles_in_1us = ceil(clk_get_rate(clki->clk), HZ_PER_MHZ); break; } } +set_core_clk_ctrl: err = ufshcd_dme_get(hba, UIC_ARG_MIB(DME_VS_CORE_CLK_CTRL), &core_clk_ctrl_reg); @@ -1473,13 +1493,13 @@ static int ufs_qcom_clk_scale_up_pre_change(struct ufs_hba *hba, unsigned long f { int ret; - ret = ufs_qcom_cfg_timers(hba, true); + ret = ufs_qcom_cfg_timers(hba, true, freq); if (ret) { dev_err(hba->dev, "%s ufs cfg timer failed\n", __func__); return ret; } /* set unipro core clock attributes and clear clock divider */ - return ufs_qcom_set_core_clk_ctrl(hba, freq); + return ufs_qcom_set_core_clk_ctrl(hba, true, freq); } static int ufs_qcom_clk_scale_up_post_change(struct ufs_hba *hba) @@ -1510,8 +1530,15 @@ static int ufs_qcom_clk_scale_down_pre_change(struct ufs_hba *hba) static int ufs_qcom_clk_scale_down_post_change(struct ufs_hba *hba, unsigned long freq) { + int ret; + + ret = ufs_qcom_cfg_timers(hba, false, freq); + if (ret) { + dev_err(hba->dev, "%s: ufs_qcom_cfg_timers() failed\n", __func__); + return ret; + } /* set unipro core clock attributes and clear clock divider */ - return ufs_qcom_set_core_clk_ctrl(hba, freq); + return ufs_qcom_set_core_clk_ctrl(hba, false, freq); } static int ufs_qcom_clk_scale_notify(struct ufs_hba *hba, bool scale_up, @@ -2092,11 +2119,53 @@ static int ufs_qcom_config_esi(struct ufs_hba *hba) return 0; } +static unsigned long ufs_qcom_opp_freq_to_clk_freq(struct ufs_hba *hba, + unsigned long freq, char *name) +{ + struct ufs_clk_info *clki; + struct dev_pm_opp *opp; + unsigned long clk_freq; + int idx = 0; + bool found = false; + + opp = dev_pm_opp_find_freq_exact_indexed(hba->dev, freq, 0, true); + if (IS_ERR(opp)) { + dev_err(hba->dev, "Failed to find OPP for exact frequency %lu\n", freq); + return 0; + } + + list_for_each_entry(clki, &hba->clk_list_head, list) { + if (!strcmp(clki->name, name)) { + found = true; + break; + } + + idx++; + } + + if (!found) { + dev_err(hba->dev, "Failed to find clock '%s' in clk list\n", name); + dev_pm_opp_put(opp); + return 0; + } + + clk_freq = dev_pm_opp_get_freq_indexed(opp, idx); + + dev_pm_opp_put(opp); + + return clk_freq; +} + static u32 ufs_qcom_freq_to_gear_speed(struct ufs_hba *hba, unsigned long freq) { - u32 gear = 0; + u32 gear = UFS_HS_DONT_CHANGE; + unsigned long unipro_freq; - switch (freq) { + if (!hba->use_pm_opp) + return gear; + + unipro_freq = ufs_qcom_opp_freq_to_clk_freq(hba, freq, "core_clk_unipro"); + switch (unipro_freq) { case 403000000: gear = UFS_HS_G5; break; @@ -2116,10 +2185,10 @@ static u32 ufs_qcom_freq_to_gear_speed(struct ufs_hba *hba, unsigned long freq) break; default: dev_err(hba->dev, "%s: Unsupported clock freq : %lu\n", __func__, freq); - break; + return UFS_HS_DONT_CHANGE; } - return gear; + return min_t(u32, gear, hba->max_pwr_info.info.gear_rx); } /* diff --git a/drivers/usb/atm/cxacru.c b/drivers/usb/atm/cxacru.c index b7f940486414..a12ab90b3db7 100644 --- a/drivers/usb/atm/cxacru.c +++ b/drivers/usb/atm/cxacru.c @@ -582,7 +582,7 @@ struct cxacru_timer { static void cxacru_timeout_kill(struct timer_list *t) { - struct cxacru_timer *timer = from_timer(timer, t, timer); + struct cxacru_timer *timer = timer_container_of(timer, t, timer); usb_unlink_urb(timer->urb); } diff --git a/drivers/usb/atm/speedtch.c b/drivers/usb/atm/speedtch.c index 27e3d35ee7dd..773ac2725532 100644 --- a/drivers/usb/atm/speedtch.c +++ b/drivers/usb/atm/speedtch.c @@ -559,8 +559,9 @@ static void speedtch_check_status(struct work_struct *work) static void speedtch_status_poll(struct timer_list *t) { - struct speedtch_instance_data *instance = from_timer(instance, t, - status_check_timer); + struct speedtch_instance_data *instance = timer_container_of(instance, + t, + status_check_timer); schedule_work(&instance->status_check_work); @@ -573,8 +574,9 @@ static void speedtch_status_poll(struct timer_list *t) static void speedtch_resubmit_int(struct timer_list *t) { - struct speedtch_instance_data *instance = from_timer(instance, t, - resubmit_timer); + struct speedtch_instance_data *instance = timer_container_of(instance, + t, + resubmit_timer); struct urb *int_urb = instance->int_urb; int ret; diff --git a/drivers/usb/atm/usbatm.c b/drivers/usb/atm/usbatm.c index a6a05e85ef8c..5f3ad9a99d9e 100644 --- a/drivers/usb/atm/usbatm.c +++ b/drivers/usb/atm/usbatm.c @@ -993,7 +993,7 @@ static int usbatm_heavy_init(struct usbatm_data *instance) static void usbatm_tasklet_schedule(struct timer_list *t) { - struct usbatm_channel *channel = from_timer(channel, t, delay); + struct usbatm_channel *channel = timer_container_of(channel, t, delay); tasklet_schedule(&channel->tasklet); } diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c index a63c793bac21..c22de97432a0 100644 --- a/drivers/usb/core/hcd.c +++ b/drivers/usb/core/hcd.c @@ -775,7 +775,7 @@ EXPORT_SYMBOL_GPL(usb_hcd_poll_rh_status); /* timer callback */ static void rh_timer_func (struct timer_list *t) { - struct usb_hcd *_hcd = from_timer(_hcd, t, rh_timer); + struct usb_hcd *_hcd = timer_container_of(_hcd, t, rh_timer); usb_hcd_poll_rh_status(_hcd); } diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c index 416af6d76374..770d1e91183c 100644 --- a/drivers/usb/core/hub.c +++ b/drivers/usb/core/hub.c @@ -697,7 +697,7 @@ static void hub_resubmit_irq_urb(struct usb_hub *hub) static void hub_retry_irq_urb(struct timer_list *t) { - struct usb_hub *hub = from_timer(hub, t, irq_urb_retry); + struct usb_hub *hub = timer_container_of(hub, t, irq_urb_retry); hub_resubmit_irq_urb(hub); } diff --git a/drivers/usb/dwc2/hcd.c b/drivers/usb/dwc2/hcd.c index 60ef8092259a..30eb8506617c 100644 --- a/drivers/usb/dwc2/hcd.c +++ b/drivers/usb/dwc2/hcd.c @@ -3227,7 +3227,7 @@ host: static void dwc2_wakeup_detected(struct timer_list *t) { - struct dwc2_hsotg *hsotg = from_timer(hsotg, t, wkp_timer); + struct dwc2_hsotg *hsotg = timer_container_of(hsotg, t, wkp_timer); u32 hprt0; dev_dbg(hsotg->dev, "%s()\n", __func__); diff --git a/drivers/usb/dwc2/hcd_queue.c b/drivers/usb/dwc2/hcd_queue.c index b0098792dd22..904fe0632b34 100644 --- a/drivers/usb/dwc2/hcd_queue.c +++ b/drivers/usb/dwc2/hcd_queue.c @@ -1215,7 +1215,7 @@ static void dwc2_do_unreserve(struct dwc2_hsotg *hsotg, struct dwc2_qh *qh) */ static void dwc2_unreserve_timer_fn(struct timer_list *t) { - struct dwc2_qh *qh = from_timer(qh, t, unreserve_timer); + struct dwc2_qh *qh = timer_container_of(qh, t, unreserve_timer); struct dwc2_hsotg *hsotg = qh->hsotg; unsigned long flags; diff --git a/drivers/usb/gadget/udc/at91_udc.c b/drivers/usb/gadget/udc/at91_udc.c index aa4c61094dc6..42b94d858e37 100644 --- a/drivers/usb/gadget/udc/at91_udc.c +++ b/drivers/usb/gadget/udc/at91_udc.c @@ -1541,7 +1541,7 @@ static void at91_vbus_timer_work(struct work_struct *work) static void at91_vbus_timer(struct timer_list *t) { - struct at91_udc *udc = from_timer(udc, t, vbus_timer); + struct at91_udc *udc = timer_container_of(udc, t, vbus_timer); /* * If we are polling vbus it is likely that the gpio is on an diff --git a/drivers/usb/gadget/udc/dummy_hcd.c b/drivers/usb/gadget/udc/dummy_hcd.c index 4f1b5db51dda..27c9699365ab 100644 --- a/drivers/usb/gadget/udc/dummy_hcd.c +++ b/drivers/usb/gadget/udc/dummy_hcd.c @@ -1787,7 +1787,8 @@ static int handle_control_request(struct dummy_hcd *dum_hcd, struct urb *urb, */ static enum hrtimer_restart dummy_timer(struct hrtimer *t) { - struct dummy_hcd *dum_hcd = from_timer(dum_hcd, t, timer); + struct dummy_hcd *dum_hcd = timer_container_of(dum_hcd, t, + timer); struct dummy *dum = dum_hcd->dum; struct urbp *urbp, *tmp; unsigned long flags; diff --git a/drivers/usb/gadget/udc/m66592-udc.c b/drivers/usb/gadget/udc/m66592-udc.c index a938b2af0944..715791737499 100644 --- a/drivers/usb/gadget/udc/m66592-udc.c +++ b/drivers/usb/gadget/udc/m66592-udc.c @@ -1261,7 +1261,7 @@ static irqreturn_t m66592_irq(int irq, void *_m66592) static void m66592_timer(struct timer_list *t) { - struct m66592 *m66592 = from_timer(m66592, t, timer); + struct m66592 *m66592 = timer_container_of(m66592, t, timer); unsigned long flags; u16 tmp; diff --git a/drivers/usb/gadget/udc/omap_udc.c b/drivers/usb/gadget/udc/omap_udc.c index c93ea210c17c..062bf2b57d2e 100644 --- a/drivers/usb/gadget/udc/omap_udc.c +++ b/drivers/usb/gadget/udc/omap_udc.c @@ -1860,7 +1860,7 @@ static irqreturn_t omap_udc_irq(int irq, void *_udc) static void pio_out_timer(struct timer_list *t) { - struct omap_ep *ep = from_timer(ep, t, timer); + struct omap_ep *ep = timer_container_of(ep, t, timer); unsigned long flags; u16 stat_flg; diff --git a/drivers/usb/gadget/udc/pxa25x_udc.c b/drivers/usb/gadget/udc/pxa25x_udc.c index 24eb1ae78e45..f5d09a91e554 100644 --- a/drivers/usb/gadget/udc/pxa25x_udc.c +++ b/drivers/usb/gadget/udc/pxa25x_udc.c @@ -1574,7 +1574,7 @@ static inline void clear_ep_state (struct pxa25x_udc *dev) static void udc_watchdog(struct timer_list *t) { - struct pxa25x_udc *dev = from_timer(dev, t, timer); + struct pxa25x_udc *dev = timer_container_of(dev, t, timer); local_irq_disable(); if (dev->ep0state == EP0_STALL diff --git a/drivers/usb/gadget/udc/r8a66597-udc.c b/drivers/usb/gadget/udc/r8a66597-udc.c index 6c1d15b2250c..e5c2630e3711 100644 --- a/drivers/usb/gadget/udc/r8a66597-udc.c +++ b/drivers/usb/gadget/udc/r8a66597-udc.c @@ -1516,7 +1516,7 @@ static irqreturn_t r8a66597_irq(int irq, void *_r8a66597) static void r8a66597_timer(struct timer_list *t) { - struct r8a66597 *r8a66597 = from_timer(r8a66597, t, timer); + struct r8a66597 *r8a66597 = timer_container_of(r8a66597, t, timer); unsigned long flags; u16 tmp; diff --git a/drivers/usb/host/ehci-platform.c b/drivers/usb/host/ehci-platform.c index 150d2542cef0..6aab45c8525c 100644 --- a/drivers/usb/host/ehci-platform.c +++ b/drivers/usb/host/ehci-platform.c @@ -198,7 +198,8 @@ static void quirk_poll_work(struct work_struct *work) static void quirk_poll_timer(struct timer_list *t) { - struct ehci_platform_priv *priv = from_timer(priv, t, poll_timer); + struct ehci_platform_priv *priv = timer_container_of(priv, t, + poll_timer); struct ehci_hcd *ehci = container_of((void *)priv, struct ehci_hcd, priv); diff --git a/drivers/usb/host/ohci-hcd.c b/drivers/usb/host/ohci-hcd.c index c7784bf8101d..9c7f3008646e 100644 --- a/drivers/usb/host/ohci-hcd.c +++ b/drivers/usb/host/ohci-hcd.c @@ -746,7 +746,8 @@ static int ohci_start(struct usb_hcd *hcd) */ static void io_watchdog_func(struct timer_list *t) { - struct ohci_hcd *ohci = from_timer(ohci, t, io_watchdog); + struct ohci_hcd *ohci = timer_container_of(ohci, t, + io_watchdog); bool takeback_all_pending = false; u32 status; u32 head; diff --git a/drivers/usb/host/oxu210hp-hcd.c b/drivers/usb/host/oxu210hp-hcd.c index d75b1b9b4db0..6b7c73eff081 100644 --- a/drivers/usb/host/oxu210hp-hcd.c +++ b/drivers/usb/host/oxu210hp-hcd.c @@ -2955,7 +2955,7 @@ static irqreturn_t oxu_irq(struct usb_hcd *hcd) static void oxu_watchdog(struct timer_list *t) { - struct oxu_hcd *oxu = from_timer(oxu, t, watchdog); + struct oxu_hcd *oxu = timer_container_of(oxu, t, watchdog); unsigned long flags; spin_lock_irqsave(&oxu->lock, flags); diff --git a/drivers/usb/host/r8a66597-hcd.c b/drivers/usb/host/r8a66597-hcd.c index 67e472116d11..d21a03cf5c17 100644 --- a/drivers/usb/host/r8a66597-hcd.c +++ b/drivers/usb/host/r8a66597-hcd.c @@ -1720,7 +1720,8 @@ static void r8a66597_root_hub_control(struct r8a66597 *r8a66597, int port) static void r8a66597_interval_timer(struct timer_list *t) { - struct r8a66597_timers *timers = from_timer(timers, t, interval); + struct r8a66597_timers *timers = timer_container_of(timers, t, + interval); struct r8a66597 *r8a66597 = timers->r8a66597; unsigned long flags; u16 pipenum; @@ -1744,7 +1745,7 @@ static void r8a66597_interval_timer(struct timer_list *t) static void r8a66597_td_timer(struct timer_list *t) { - struct r8a66597_timers *timers = from_timer(timers, t, td); + struct r8a66597_timers *timers = timer_container_of(timers, t, td); struct r8a66597 *r8a66597 = timers->r8a66597; unsigned long flags; u16 pipenum; @@ -1798,7 +1799,7 @@ static void r8a66597_td_timer(struct timer_list *t) static void r8a66597_timer(struct timer_list *t) { - struct r8a66597 *r8a66597 = from_timer(r8a66597, t, rh_timer); + struct r8a66597 *r8a66597 = timer_container_of(r8a66597, t, rh_timer); unsigned long flags; int port; diff --git a/drivers/usb/host/sl811-hcd.c b/drivers/usb/host/sl811-hcd.c index 718b1b7fe366..ea3cab99c5d4 100644 --- a/drivers/usb/host/sl811-hcd.c +++ b/drivers/usb/host/sl811-hcd.c @@ -1124,7 +1124,7 @@ sl811h_hub_descriptor ( static void sl811h_timer(struct timer_list *t) { - struct sl811 *sl811 = from_timer(sl811, t, timer); + struct sl811 *sl811 = timer_container_of(sl811, t, timer); unsigned long flags; u8 irqstat; u8 signaling = sl811->ctrl1 & SL11H_CTL1MASK_FORCE; diff --git a/drivers/usb/host/uhci-q.c b/drivers/usb/host/uhci-q.c index 45a8256a665f..9480d4ff0111 100644 --- a/drivers/usb/host/uhci-q.c +++ b/drivers/usb/host/uhci-q.c @@ -91,7 +91,7 @@ static void uhci_urbp_wants_fsbr(struct uhci_hcd *uhci, struct urb_priv *urbp) static void uhci_fsbr_timeout(struct timer_list *t) { - struct uhci_hcd *uhci = from_timer(uhci, t, fsbr_timer); + struct uhci_hcd *uhci = timer_container_of(uhci, t, fsbr_timer); unsigned long flags; spin_lock_irqsave(&uhci->lock, flags); diff --git a/drivers/usb/host/xen-hcd.c b/drivers/usb/host/xen-hcd.c index 05943f2213e4..1c2a95fe41e5 100644 --- a/drivers/usb/host/xen-hcd.c +++ b/drivers/usb/host/xen-hcd.c @@ -1258,7 +1258,7 @@ static void xenhcd_disconnect(struct xenbus_device *dev) static void xenhcd_watchdog(struct timer_list *timer) { - struct xenhcd_info *info = from_timer(info, timer, watchdog); + struct xenhcd_info *info = timer_container_of(info, timer, watchdog); unsigned long flags; spin_lock_irqsave(&info->lock, flags); diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c index 803047bec3e7..4e6dbd2375c3 100644 --- a/drivers/usb/host/xhci.c +++ b/drivers/usb/host/xhci.c @@ -385,7 +385,7 @@ static void compliance_mode_recovery(struct timer_list *t) u32 temp; int i; - xhci = from_timer(xhci, t, comp_mode_recovery_timer); + xhci = timer_container_of(xhci, t, comp_mode_recovery_timer); rhub = &xhci->usb3_rhub; hcd = rhub->hcd; diff --git a/drivers/usb/isp1760/isp1760-udc.c b/drivers/usb/isp1760/isp1760-udc.c index 2af89ee28baa..65ac91d0595a 100644 --- a/drivers/usb/isp1760/isp1760-udc.c +++ b/drivers/usb/isp1760/isp1760-udc.c @@ -1423,7 +1423,7 @@ static irqreturn_t isp1760_udc_irq(int irq, void *dev) static void isp1760_udc_vbus_poll(struct timer_list *t) { - struct isp1760_udc *udc = from_timer(udc, t, vbus_timer); + struct isp1760_udc *udc = timer_container_of(udc, t, vbus_timer); unsigned long flags; spin_lock_irqsave(&udc->lock, flags); diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c index 7c2041f61cde..5c92c8d8e283 100644 --- a/drivers/usb/misc/usbtest.c +++ b/drivers/usb/misc/usbtest.c @@ -592,7 +592,7 @@ struct sg_timeout { static void sg_timeout(struct timer_list *t) { - struct sg_timeout *timeout = from_timer(timeout, t, timer); + struct sg_timeout *timeout = timer_container_of(timeout, t, timer); usb_sg_cancel(timeout->req); } diff --git a/drivers/usb/musb/da8xx.c b/drivers/usb/musb/da8xx.c index eebb24ab3ec8..4209f438ba18 100644 --- a/drivers/usb/musb/da8xx.c +++ b/drivers/usb/musb/da8xx.c @@ -123,7 +123,8 @@ static void da8xx_musb_set_vbus(struct musb *musb, int is_on) static void otg_timer(struct timer_list *t) { - struct musb *musb = from_timer(musb, t, dev_timer); + struct musb *musb = timer_container_of(musb, t, + dev_timer); void __iomem *mregs = musb->mregs; u8 devctl; unsigned long flags; diff --git a/drivers/usb/musb/mpfs.c b/drivers/usb/musb/mpfs.c index 020348a98514..587127abd30a 100644 --- a/drivers/usb/musb/mpfs.c +++ b/drivers/usb/musb/mpfs.c @@ -91,7 +91,8 @@ static void mpfs_musb_set_vbus(struct musb *musb, int is_on) static void otg_timer(struct timer_list *t) { - struct musb *musb = from_timer(musb, t, dev_timer); + struct musb *musb = timer_container_of(musb, t, + dev_timer); void __iomem *mregs = musb->mregs; u8 devctl; unsigned long flags; diff --git a/drivers/usb/musb/musb_core.c b/drivers/usb/musb/musb_core.c index cbbb27178024..c7234b236971 100644 --- a/drivers/usb/musb/musb_core.c +++ b/drivers/usb/musb/musb_core.c @@ -596,7 +596,7 @@ void musb_load_testpacket(struct musb *musb) */ static void musb_otg_timer_func(struct timer_list *t) { - struct musb *musb = from_timer(musb, t, otg_timer); + struct musb *musb = timer_container_of(musb, t, otg_timer); unsigned long flags; spin_lock_irqsave(&musb->lock, flags); diff --git a/drivers/usb/musb/musb_dsps.c b/drivers/usb/musb/musb_dsps.c index e5e813f97fac..12f587ab8511 100644 --- a/drivers/usb/musb/musb_dsps.c +++ b/drivers/usb/musb/musb_dsps.c @@ -278,7 +278,7 @@ static int dsps_check_status(struct musb *musb, void *unused) static void otg_timer(struct timer_list *t) { - struct musb *musb = from_timer(musb, t, dev_timer); + struct musb *musb = timer_container_of(musb, t, dev_timer); struct device *dev = musb->controller; unsigned long flags; int err; diff --git a/drivers/usb/musb/tusb6010.c b/drivers/usb/musb/tusb6010.c index abd2472da7f7..1d9571f24a56 100644 --- a/drivers/usb/musb/tusb6010.c +++ b/drivers/usb/musb/tusb6010.c @@ -457,7 +457,7 @@ static int tusb_musb_vbus_status(struct musb *musb) static void musb_do_idle(struct timer_list *t) { - struct musb *musb = from_timer(musb, t, dev_timer); + struct musb *musb = timer_container_of(musb, t, dev_timer); unsigned long flags; spin_lock_irqsave(&musb->lock, flags); diff --git a/drivers/usb/serial/garmin_gps.c b/drivers/usb/serial/garmin_gps.c index b97ba8ed6801..614de0c2f5f2 100644 --- a/drivers/usb/serial/garmin_gps.c +++ b/drivers/usb/serial/garmin_gps.c @@ -1357,7 +1357,8 @@ static void garmin_unthrottle(struct tty_struct *tty) */ static void timeout_handler(struct timer_list *t) { - struct garmin_data *garmin_data_p = from_timer(garmin_data_p, t, timer); + struct garmin_data *garmin_data_p = timer_container_of(garmin_data_p, + t, timer); /* send the next queued packet to the tty port */ if (garmin_data_p->mode == MODE_NATIVE) diff --git a/drivers/usb/serial/mos7840.c b/drivers/usb/serial/mos7840.c index 93710b762893..9e9aca271c0a 100644 --- a/drivers/usb/serial/mos7840.c +++ b/drivers/usb/serial/mos7840.c @@ -383,7 +383,7 @@ static void mos7840_set_led_sync(struct usb_serial_port *port, __u16 reg, static void mos7840_led_off(struct timer_list *t) { - struct moschip_port *mcs = from_timer(mcs, t, led_timer1); + struct moschip_port *mcs = timer_container_of(mcs, t, led_timer1); /* Turn off LED */ mos7840_set_led_async(mcs, 0x0300, MODEM_CONTROL_REGISTER); @@ -393,7 +393,7 @@ static void mos7840_led_off(struct timer_list *t) static void mos7840_led_flag_off(struct timer_list *t) { - struct moschip_port *mcs = from_timer(mcs, t, led_timer2); + struct moschip_port *mcs = timer_container_of(mcs, t, led_timer2); clear_bit_unlock(MOS7840_FLAG_LED_BUSY, &mcs->flags); } diff --git a/drivers/usb/storage/realtek_cr.c b/drivers/usb/storage/realtek_cr.c index b387863c245f..c18dfa2ca034 100644 --- a/drivers/usb/storage/realtek_cr.c +++ b/drivers/usb/storage/realtek_cr.c @@ -754,7 +754,8 @@ static void rts51x_modi_suspend_timer(struct rts51x_chip *chip) static void rts51x_suspend_timer_fn(struct timer_list *t) { - struct rts51x_chip *chip = from_timer(chip, t, rts51x_suspend_timer); + struct rts51x_chip *chip = timer_container_of(chip, t, + rts51x_suspend_timer); struct us_data *us = chip->us; switch (rts51x_get_stat(chip)) { diff --git a/drivers/usb/usbip/vudc_transfer.c b/drivers/usb/usbip/vudc_transfer.c index 7e801fee33bf..a4f02ea3e3ef 100644 --- a/drivers/usb/usbip/vudc_transfer.c +++ b/drivers/usb/usbip/vudc_transfer.c @@ -301,7 +301,7 @@ top: static void v_timer(struct timer_list *t) { - struct vudc *udc = from_timer(udc, t, tr_timer.timer); + struct vudc *udc = timer_container_of(udc, t, tr_timer.timer); struct transfer_timer *timer = &udc->tr_timer; struct urbp *urb_p, *tmp; unsigned long flags; diff --git a/drivers/video/fbdev/aty/radeon_base.c b/drivers/video/fbdev/aty/radeon_base.c index c6c4753f6415..0eef8c6b98c8 100644 --- a/drivers/video/fbdev/aty/radeon_base.c +++ b/drivers/video/fbdev/aty/radeon_base.c @@ -1443,7 +1443,7 @@ static void radeon_write_pll_regs(struct radeonfb_info *rinfo, struct radeon_reg */ static void radeon_lvds_timer_func(struct timer_list *t) { - struct radeonfb_info *rinfo = from_timer(rinfo, t, lvds_timer); + struct radeonfb_info *rinfo = timer_container_of(rinfo, t, lvds_timer); radeon_engine_idle(); diff --git a/drivers/virt/vboxguest/vboxguest_core.c b/drivers/virt/vboxguest/vboxguest_core.c index f1674f3ed923..b177a534b6a4 100644 --- a/drivers/virt/vboxguest/vboxguest_core.c +++ b/drivers/virt/vboxguest/vboxguest_core.c @@ -419,7 +419,7 @@ static void vbg_balloon_work(struct work_struct *work) */ static void vbg_heartbeat_timer(struct timer_list *t) { - struct vbg_dev *gdev = from_timer(gdev, t, heartbeat_timer); + struct vbg_dev *gdev = timer_container_of(gdev, t, heartbeat_timer); vbg_req_perform(gdev, gdev->guest_heartbeat_req); mod_timer(&gdev->heartbeat_timer, diff --git a/drivers/watchdog/at91sam9_wdt.c b/drivers/watchdog/at91sam9_wdt.c index 1b47a2fc7d17..aba66b8e9d03 100644 --- a/drivers/watchdog/at91sam9_wdt.c +++ b/drivers/watchdog/at91sam9_wdt.c @@ -119,7 +119,7 @@ static inline void at91_wdt_reset(struct at91wdt *wdt) */ static void at91_ping(struct timer_list *t) { - struct at91wdt *wdt = from_timer(wdt, t, timer); + struct at91wdt *wdt = timer_container_of(wdt, t, timer); if (time_before(jiffies, wdt->next_heartbeat) || !watchdog_active(&wdt->wdd)) { at91_wdt_reset(wdt); diff --git a/drivers/watchdog/bcm47xx_wdt.c b/drivers/watchdog/bcm47xx_wdt.c index 4c0951307421..e13ec0975bef 100644 --- a/drivers/watchdog/bcm47xx_wdt.c +++ b/drivers/watchdog/bcm47xx_wdt.c @@ -105,7 +105,7 @@ static const struct watchdog_ops bcm47xx_wdt_hard_ops = { static void bcm47xx_wdt_soft_timer_tick(struct timer_list *t) { - struct bcm47xx_wdt *wdt = from_timer(wdt, t, soft_timer); + struct bcm47xx_wdt *wdt = timer_container_of(wdt, t, soft_timer); u32 next_tick = min(wdt->wdd.timeout * 1000, wdt->max_timer_ms); if (!atomic_dec_and_test(&wdt->soft_ticks)) { diff --git a/drivers/watchdog/lpc18xx_wdt.c b/drivers/watchdog/lpc18xx_wdt.c index 28e3fc0df4c6..3e8e80bbcb93 100644 --- a/drivers/watchdog/lpc18xx_wdt.c +++ b/drivers/watchdog/lpc18xx_wdt.c @@ -77,7 +77,8 @@ static int lpc18xx_wdt_feed(struct watchdog_device *wdt_dev) static void lpc18xx_wdt_timer_feed(struct timer_list *t) { - struct lpc18xx_wdt_dev *lpc18xx_wdt = from_timer(lpc18xx_wdt, t, timer); + struct lpc18xx_wdt_dev *lpc18xx_wdt = timer_container_of(lpc18xx_wdt, + t, timer); struct watchdog_device *wdt_dev = &lpc18xx_wdt->wdt_dev; lpc18xx_wdt_feed(wdt_dev); diff --git a/drivers/watchdog/shwdt.c b/drivers/watchdog/shwdt.c index 95af9ad94d16..719f100aae60 100644 --- a/drivers/watchdog/shwdt.c +++ b/drivers/watchdog/shwdt.c @@ -173,7 +173,7 @@ static int sh_wdt_set_heartbeat(struct watchdog_device *wdt_dev, unsigned t) static void sh_wdt_ping(struct timer_list *t) { - struct sh_wdt *wdt = from_timer(wdt, t, timer); + struct sh_wdt *wdt = timer_container_of(wdt, t, timer); unsigned long flags; spin_lock_irqsave(&wdt->lock, flags); diff --git a/fs/bcachefs/bcachefs.h b/fs/bcachefs/bcachefs.h index 3651a296d506..5a1cede2febf 100644 --- a/fs/bcachefs/bcachefs.h +++ b/fs/bcachefs/bcachefs.h @@ -296,7 +296,6 @@ do { \ #define bch2_fmt(_c, fmt) bch2_log_msg(_c, fmt "\n") void bch2_print_str(struct bch_fs *, const char *, const char *); -void bch2_print_str_nonblocking(struct bch_fs *, const char *, const char *); __printf(2, 3) void bch2_print_opts(struct bch_opts *, const char *, ...); diff --git a/fs/bcachefs/btree_gc.c b/fs/bcachefs/btree_gc.c index 9ddcbe1bda78..e92cf3928c63 100644 --- a/fs/bcachefs/btree_gc.c +++ b/fs/bcachefs/btree_gc.c @@ -397,7 +397,11 @@ again: continue; } - ret = btree_check_node_boundaries(trans, b, prev, cur, pulled_from_scan); + ret = lockrestart_do(trans, + btree_check_node_boundaries(trans, b, prev, cur, pulled_from_scan)); + if (ret < 0) + goto err; + if (ret == DID_FILL_FROM_SCAN) { new_pass = true; ret = 0; @@ -438,7 +442,8 @@ again: if (!ret && !IS_ERR_OR_NULL(prev)) { BUG_ON(cur); - ret = btree_repair_node_end(trans, b, prev, pulled_from_scan); + ret = lockrestart_do(trans, + btree_repair_node_end(trans, b, prev, pulled_from_scan)); if (ret == DID_FILL_FROM_SCAN) { new_pass = true; ret = 0; @@ -519,6 +524,46 @@ fsck_err: bch2_bkey_buf_exit(&prev_k, c); bch2_bkey_buf_exit(&cur_k, c); printbuf_exit(&buf); + bch_err_fn(c, ret); + return ret; +} + +static int bch2_check_root(struct btree_trans *trans, enum btree_id i, + bool *reconstructed_root) +{ + struct bch_fs *c = trans->c; + struct btree_root *r = bch2_btree_id_root(c, i); + struct printbuf buf = PRINTBUF; + int ret = 0; + + bch2_btree_id_to_text(&buf, i); + + if (r->error) { + bch_info(c, "btree root %s unreadable, must recover from scan", buf.buf); + + r->alive = false; + r->error = 0; + + if (!bch2_btree_has_scanned_nodes(c, i)) { + __fsck_err(trans, + FSCK_CAN_FIX|(!btree_id_important(i) ? FSCK_AUTOFIX : 0), + btree_root_unreadable_and_scan_found_nothing, + "no nodes found for btree %s, continue?", buf.buf); + bch2_btree_root_alloc_fake_trans(trans, i, 0); + } else { + bch2_btree_root_alloc_fake_trans(trans, i, 1); + bch2_shoot_down_journal_keys(c, i, 1, BTREE_MAX_DEPTH, POS_MIN, SPOS_MAX); + ret = bch2_get_scanned_nodes(c, i, 0, POS_MIN, SPOS_MAX); + if (ret) + goto err; + } + + *reconstructed_root = true; + } +err: +fsck_err: + printbuf_exit(&buf); + bch_err_fn(c, ret); return ret; } @@ -526,42 +571,18 @@ int bch2_check_topology(struct bch_fs *c) { struct btree_trans *trans = bch2_trans_get(c); struct bpos pulled_from_scan = POS_MIN; - struct printbuf buf = PRINTBUF; int ret = 0; bch2_trans_srcu_unlock(trans); for (unsigned i = 0; i < btree_id_nr_alive(c) && !ret; i++) { - struct btree_root *r = bch2_btree_id_root(c, i); bool reconstructed_root = false; +recover: + ret = lockrestart_do(trans, bch2_check_root(trans, i, &reconstructed_root)); + if (ret) + break; - printbuf_reset(&buf); - bch2_btree_id_to_text(&buf, i); - - if (r->error) { -reconstruct_root: - bch_info(c, "btree root %s unreadable, must recover from scan", buf.buf); - - r->alive = false; - r->error = 0; - - if (!bch2_btree_has_scanned_nodes(c, i)) { - __fsck_err(trans, - FSCK_CAN_FIX|(!btree_id_important(i) ? FSCK_AUTOFIX : 0), - btree_root_unreadable_and_scan_found_nothing, - "no nodes found for btree %s, continue?", buf.buf); - bch2_btree_root_alloc_fake_trans(trans, i, 0); - } else { - bch2_btree_root_alloc_fake_trans(trans, i, 1); - bch2_shoot_down_journal_keys(c, i, 1, BTREE_MAX_DEPTH, POS_MIN, SPOS_MAX); - ret = bch2_get_scanned_nodes(c, i, 0, POS_MIN, SPOS_MAX); - if (ret) - break; - } - - reconstructed_root = true; - } - + struct btree_root *r = bch2_btree_id_root(c, i); struct btree *b = r->b; btree_node_lock_nopath_nofail(trans, &b->c, SIX_LOCK_read); @@ -575,17 +596,21 @@ reconstruct_root: r->b = NULL; - if (!reconstructed_root) - goto reconstruct_root; + if (!reconstructed_root) { + r->error = -EIO; + goto recover; + } + struct printbuf buf = PRINTBUF; + bch2_btree_id_to_text(&buf, i); bch_err(c, "empty btree root %s", buf.buf); + printbuf_exit(&buf); bch2_btree_root_alloc_fake_trans(trans, i, 0); r->alive = false; ret = 0; } } -fsck_err: - printbuf_exit(&buf); + bch2_trans_put(trans); return ret; } diff --git a/fs/bcachefs/btree_io.c b/fs/bcachefs/btree_io.c index 57eff3012a7b..d8f3c4c65e90 100644 --- a/fs/bcachefs/btree_io.c +++ b/fs/bcachefs/btree_io.c @@ -741,16 +741,22 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, BCH_VERSION_MAJOR(version), BCH_VERSION_MINOR(version)); - if (btree_err_on(version < c->sb.version_min, + if (c->recovery.curr_pass != BCH_RECOVERY_PASS_scan_for_btree_nodes && + btree_err_on(version < c->sb.version_min, -BCH_ERR_btree_node_read_err_fixable, c, NULL, b, i, NULL, btree_node_bset_older_than_sb_min, "bset version %u older than superblock version_min %u", version, c->sb.version_min)) { - mutex_lock(&c->sb_lock); - c->disk_sb.sb->version_min = cpu_to_le16(version); - bch2_write_super(c); - mutex_unlock(&c->sb_lock); + if (bch2_version_compatible(version)) { + mutex_lock(&c->sb_lock); + c->disk_sb.sb->version_min = cpu_to_le16(version); + bch2_write_super(c); + mutex_unlock(&c->sb_lock); + } else { + /* We have no idea what's going on: */ + i->version = cpu_to_le16(c->sb.version); + } } if (btree_err_on(BCH_VERSION_MAJOR(version) > @@ -1045,6 +1051,7 @@ got_good_key: le16_add_cpu(&i->u64s, -next_good_key); memmove_u64s_down(k, (u64 *) k + next_good_key, (u64 *) vstruct_end(i) - (u64 *) k); set_btree_node_need_rewrite(b); + set_btree_node_need_rewrite_error(b); } fsck_err: printbuf_exit(&buf); @@ -1305,6 +1312,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, (u64 *) vstruct_end(i) - (u64 *) k); set_btree_bset_end(b, b->set); set_btree_node_need_rewrite(b); + set_btree_node_need_rewrite_error(b); continue; } if (ret) @@ -1329,12 +1337,16 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, bkey_for_each_ptr(bch2_bkey_ptrs(bkey_i_to_s(&b->key)), ptr) { struct bch_dev *ca2 = bch2_dev_rcu(c, ptr->dev); - if (!ca2 || ca2->mi.state != BCH_MEMBER_STATE_rw) + if (!ca2 || ca2->mi.state != BCH_MEMBER_STATE_rw) { set_btree_node_need_rewrite(b); + set_btree_node_need_rewrite_degraded(b); + } } - if (!ptr_written) + if (!ptr_written) { set_btree_node_need_rewrite(b); + set_btree_node_need_rewrite_ptr_written_zero(b); + } fsck_err: mempool_free(iter, &c->fill_iter); printbuf_exit(&buf); diff --git a/fs/bcachefs/btree_locking.c b/fs/bcachefs/btree_locking.c index 47035aae232e..91a51aef82f1 100644 --- a/fs/bcachefs/btree_locking.c +++ b/fs/bcachefs/btree_locking.c @@ -213,7 +213,7 @@ static noinline __noreturn void break_cycle_fail(struct lock_graph *g) prt_newline(&buf); } - bch2_print_str_nonblocking(g->g->trans->c, KERN_ERR, buf.buf); + bch2_print_str(g->g->trans->c, KERN_ERR, buf.buf); printbuf_exit(&buf); BUG(); } diff --git a/fs/bcachefs/btree_locking.h b/fs/bcachefs/btree_locking.h index 9adca77e2580..f2173a3316f4 100644 --- a/fs/bcachefs/btree_locking.h +++ b/fs/bcachefs/btree_locking.h @@ -417,8 +417,10 @@ static inline void btree_path_set_should_be_locked(struct btree_trans *trans, st EBUG_ON(!btree_node_locked(path, path->level)); EBUG_ON(path->uptodate); - path->should_be_locked = true; - trace_btree_path_should_be_locked(trans, path); + if (!path->should_be_locked) { + path->should_be_locked = true; + trace_btree_path_should_be_locked(trans, path); + } } static inline void __btree_path_set_level_up(struct btree_trans *trans, diff --git a/fs/bcachefs/btree_types.h b/fs/bcachefs/btree_types.h index c61c4171ae50..3aa4a602bd02 100644 --- a/fs/bcachefs/btree_types.h +++ b/fs/bcachefs/btree_types.h @@ -617,6 +617,9 @@ enum btree_write_type { x(dying) \ x(fake) \ x(need_rewrite) \ + x(need_rewrite_error) \ + x(need_rewrite_degraded) \ + x(need_rewrite_ptr_written_zero) \ x(never_write) \ x(pinned) @@ -641,6 +644,32 @@ static inline void clear_btree_node_ ## flag(struct btree *b) \ BTREE_FLAGS() #undef x +#define BTREE_NODE_REWRITE_REASON() \ + x(none) \ + x(unknown) \ + x(error) \ + x(degraded) \ + x(ptr_written_zero) + +enum btree_node_rewrite_reason { +#define x(n) BTREE_NODE_REWRITE_##n, + BTREE_NODE_REWRITE_REASON() +#undef x +}; + +static inline enum btree_node_rewrite_reason btree_node_rewrite_reason(struct btree *b) +{ + if (btree_node_need_rewrite_ptr_written_zero(b)) + return BTREE_NODE_REWRITE_ptr_written_zero; + if (btree_node_need_rewrite_degraded(b)) + return BTREE_NODE_REWRITE_degraded; + if (btree_node_need_rewrite_error(b)) + return BTREE_NODE_REWRITE_error; + if (btree_node_need_rewrite(b)) + return BTREE_NODE_REWRITE_unknown; + return BTREE_NODE_REWRITE_none; +} + static inline struct btree_write *btree_current_write(struct btree *b) { return b->writes + btree_node_write_idx(b); diff --git a/fs/bcachefs/btree_update_interior.c b/fs/bcachefs/btree_update_interior.c index d2ecb782919b..e77584607f0d 100644 --- a/fs/bcachefs/btree_update_interior.c +++ b/fs/bcachefs/btree_update_interior.c @@ -1138,6 +1138,13 @@ static void bch2_btree_update_done(struct btree_update *as, struct btree_trans * start_time); } +static const char * const btree_node_reawrite_reason_strs[] = { +#define x(n) #n, + BTREE_NODE_REWRITE_REASON() +#undef x + NULL, +}; + static struct btree_update * bch2_btree_update_start(struct btree_trans *trans, struct btree_path *path, unsigned level_start, bool split, @@ -1232,6 +1239,15 @@ bch2_btree_update_start(struct btree_trans *trans, struct btree_path *path, list_add_tail(&as->list, &c->btree_interior_update_list); mutex_unlock(&c->btree_interior_update_lock); + struct btree *b = btree_path_node(path, path->level); + as->node_start = b->data->min_key; + as->node_end = b->data->max_key; + as->node_needed_rewrite = btree_node_rewrite_reason(b); + as->node_written = b->written; + as->node_sectors = btree_buf_bytes(b) >> 9; + as->node_remaining = __bch2_btree_u64s_remaining(b, + btree_bkey_last(b, bset_tree_last(b))); + /* * We don't want to allocate if we're in an error state, that can cause * deadlock on emergency shutdown due to open buckets getting stuck in @@ -2108,6 +2124,9 @@ int __bch2_foreground_maybe_merge(struct btree_trans *trans, if (ret) goto err; + as->node_start = prev->data->min_key; + as->node_end = next->data->max_key; + trace_and_count(c, btree_node_merge, trans, b); n = bch2_btree_node_alloc(as, trans, b->c.level); @@ -2681,9 +2700,19 @@ static void bch2_btree_update_to_text(struct printbuf *out, struct btree_update prt_str(out, " "); bch2_btree_id_to_text(out, as->btree_id); - prt_printf(out, " l=%u-%u mode=%s nodes_written=%u cl.remaining=%u journal_seq=%llu\n", + prt_printf(out, " l=%u-%u ", as->update_level_start, - as->update_level_end, + as->update_level_end); + bch2_bpos_to_text(out, as->node_start); + prt_char(out, ' '); + bch2_bpos_to_text(out, as->node_end); + prt_printf(out, "\nwritten %u/%u u64s_remaining %u need_rewrite %s", + as->node_written, + as->node_sectors, + as->node_remaining, + btree_node_reawrite_reason_strs[as->node_needed_rewrite]); + + prt_printf(out, "\nmode=%s nodes_written=%u cl.remaining=%u journal_seq=%llu\n", bch2_btree_update_modes[as->mode], as->nodes_written, closure_nr_remaining(&as->cl), diff --git a/fs/bcachefs/btree_update_interior.h b/fs/bcachefs/btree_update_interior.h index 7fe793788a79..b649c36c3fbb 100644 --- a/fs/bcachefs/btree_update_interior.h +++ b/fs/bcachefs/btree_update_interior.h @@ -57,6 +57,13 @@ struct btree_update { unsigned took_gc_lock:1; enum btree_id btree_id; + struct bpos node_start; + struct bpos node_end; + enum btree_node_rewrite_reason node_needed_rewrite; + u16 node_written; + u16 node_sectors; + u16 node_remaining; + unsigned update_level_start; unsigned update_level_end; diff --git a/fs/bcachefs/chardev.c b/fs/bcachefs/chardev.c index 2d38466eddfd..fde3c2380e28 100644 --- a/fs/bcachefs/chardev.c +++ b/fs/bcachefs/chardev.c @@ -399,7 +399,7 @@ static long bch2_ioctl_data(struct bch_fs *c, return ret; } -static long bch2_ioctl_fs_usage(struct bch_fs *c, +static noinline_for_stack long bch2_ioctl_fs_usage(struct bch_fs *c, struct bch_ioctl_fs_usage __user *user_arg) { struct bch_ioctl_fs_usage arg = {}; @@ -469,7 +469,7 @@ err: } /* obsolete, didn't allow for new data types: */ -static long bch2_ioctl_dev_usage(struct bch_fs *c, +static noinline_for_stack long bch2_ioctl_dev_usage(struct bch_fs *c, struct bch_ioctl_dev_usage __user *user_arg) { struct bch_ioctl_dev_usage arg; diff --git a/fs/bcachefs/disk_accounting.c b/fs/bcachefs/disk_accounting.c index 3d59a57a5256..f7528cd69c73 100644 --- a/fs/bcachefs/disk_accounting.c +++ b/fs/bcachefs/disk_accounting.c @@ -618,7 +618,9 @@ int bch2_gc_accounting_done(struct bch_fs *c) for (unsigned j = 0; j < nr; j++) src_v[j] -= dst_v[j]; - if (fsck_err(trans, accounting_mismatch, "%s", buf.buf)) { + bch2_trans_unlock_long(trans); + + if (fsck_err(c, accounting_mismatch, "%s", buf.buf)) { percpu_up_write(&c->mark_lock); ret = commit_do(trans, NULL, NULL, 0, bch2_disk_accounting_mod(trans, &acc_k, src_v, nr, false)); diff --git a/fs/bcachefs/error.c b/fs/bcachefs/error.c index 63951e293c47..a8ec6aae5738 100644 --- a/fs/bcachefs/error.c +++ b/fs/bcachefs/error.c @@ -69,7 +69,7 @@ static bool bch2_fs_trans_inconsistent(struct bch_fs *c, struct btree_trans *tra if (trans) bch2_trans_updates_to_text(&buf, trans); bool ret = __bch2_inconsistent_error(c, &buf); - bch2_print_str_nonblocking(c, KERN_ERR, buf.buf); + bch2_print_str(c, KERN_ERR, buf.buf); printbuf_exit(&buf); return ret; @@ -620,6 +620,9 @@ print: if (s) s->ret = ret; + + if (trans) + ret = bch2_trans_log_str(trans, bch2_sb_error_strs[err]) ?: ret; err_unlock: mutex_unlock(&c->fsck_error_msgs_lock); err: diff --git a/fs/bcachefs/fs.c b/fs/bcachefs/fs.c index 85d13f800165..3063a8ddc2df 100644 --- a/fs/bcachefs/fs.c +++ b/fs/bcachefs/fs.c @@ -2490,6 +2490,14 @@ static int bch2_fs_get_tree(struct fs_context *fc) if (ret) goto err_stop_fs; + /* + * We might be doing a RO mount because other options required it, or we + * have no alloc info and it's a small image with no room to regenerate + * it + */ + if (c->opts.read_only) + fc->sb_flags |= SB_RDONLY; + sb = sget(fc->fs_type, NULL, bch2_set_super, fc->sb_flags|SB_NOSEC, c); ret = PTR_ERR_OR_ZERO(sb); if (ret) diff --git a/fs/bcachefs/io_read.c b/fs/bcachefs/io_read.c index a77779afad01..04bbdcf58e40 100644 --- a/fs/bcachefs/io_read.c +++ b/fs/bcachefs/io_read.c @@ -343,6 +343,10 @@ static struct bch_read_bio *promote_alloc(struct btree_trans *trans, *bounce = true; *read_full = promote_full; + + if (have_io_error(failed)) + orig->self_healing = true; + return promote; nopromote: trace_io_read_nopromote(c, ret); @@ -635,12 +639,15 @@ static void bch2_rbio_retry(struct work_struct *work) prt_str(&buf, "(internal move) "); prt_str(&buf, "data read error, "); - if (!ret) + if (!ret) { prt_str(&buf, "successful retry"); - else + if (rbio->self_healing) + prt_str(&buf, ", self healing"); + } else prt_str(&buf, bch2_err_str(ret)); prt_newline(&buf); + if (!bkey_deleted(&sk.k->k)) { bch2_bkey_val_to_text(&buf, c, bkey_i_to_s_c(sk.k)); prt_newline(&buf); diff --git a/fs/bcachefs/io_read.h b/fs/bcachefs/io_read.h index 45c959018919..9c5ddbf861b3 100644 --- a/fs/bcachefs/io_read.h +++ b/fs/bcachefs/io_read.h @@ -44,6 +44,7 @@ struct bch_read_bio { have_ioref:1, narrow_crcs:1, saw_error:1, + self_healing:1, context:2; }; u16 _state; diff --git a/fs/bcachefs/movinggc.c b/fs/bcachefs/movinggc.c index 6d7b1d5f7697..27e68d470ad0 100644 --- a/fs/bcachefs/movinggc.c +++ b/fs/bcachefs/movinggc.c @@ -28,7 +28,7 @@ #include <linux/wait.h> struct buckets_in_flight { - struct rhashtable table; + struct rhashtable *table; struct move_bucket *first; struct move_bucket *last; size_t nr; @@ -98,7 +98,7 @@ out: static void move_bucket_free(struct buckets_in_flight *list, struct move_bucket *b) { - int ret = rhashtable_remove_fast(&list->table, &b->hash, + int ret = rhashtable_remove_fast(list->table, &b->hash, bch_move_bucket_params); BUG_ON(ret); kfree(b); @@ -133,7 +133,7 @@ static void move_buckets_wait(struct moving_context *ctxt, static bool bucket_in_flight(struct buckets_in_flight *list, struct move_bucket_key k) { - return rhashtable_lookup_fast(&list->table, &k, bch_move_bucket_params); + return rhashtable_lookup_fast(list->table, &k, bch_move_bucket_params); } static int bch2_copygc_get_buckets(struct moving_context *ctxt, @@ -185,7 +185,7 @@ static int bch2_copygc_get_buckets(struct moving_context *ctxt, goto err; } - ret2 = rhashtable_lookup_insert_fast(&buckets_in_flight->table, &b_i->hash, + ret2 = rhashtable_lookup_insert_fast(buckets_in_flight->table, &b_i->hash, bch_move_bucket_params); BUG_ON(ret2); @@ -350,10 +350,13 @@ static int bch2_copygc_thread(void *arg) struct buckets_in_flight buckets = {}; u64 last, wait; - int ret = rhashtable_init(&buckets.table, &bch_move_bucket_params); + buckets.table = kzalloc(sizeof(*buckets.table), GFP_KERNEL); + int ret = !buckets.table + ? -ENOMEM + : rhashtable_init(buckets.table, &bch_move_bucket_params); bch_err_msg(c, ret, "allocating copygc buckets in flight"); if (ret) - return ret; + goto err; set_freezable(); @@ -421,11 +424,12 @@ static int bch2_copygc_thread(void *arg) } move_buckets_wait(&ctxt, &buckets, true); - rhashtable_destroy(&buckets.table); + rhashtable_destroy(buckets.table); bch2_moving_ctxt_exit(&ctxt); bch2_move_stats_exit(&move_stats, c); - - return 0; +err: + kfree(buckets.table); + return ret; } void bch2_copygc_stop(struct bch_fs *c) diff --git a/fs/bcachefs/namei.c b/fs/bcachefs/namei.c index 24120037c031..779c22eb3979 100644 --- a/fs/bcachefs/namei.c +++ b/fs/bcachefs/namei.c @@ -175,6 +175,16 @@ int bch2_create_trans(struct btree_trans *trans, new_inode->bi_dir_offset = dir_offset; } + if (S_ISDIR(mode)) { + ret = bch2_maybe_propagate_has_case_insensitive(trans, + (subvol_inum) { + new_inode->bi_subvol ?: dir.subvol, + new_inode->bi_inum }, + new_inode); + if (ret) + goto err; + } + if (S_ISDIR(mode) && !new_inode->bi_subvol) new_inode->bi_depth = dir_u->bi_depth + 1; diff --git a/fs/bcachefs/rcu_pending.c b/fs/bcachefs/rcu_pending.c index bef2aa1b8bcd..b1438be9d690 100644 --- a/fs/bcachefs/rcu_pending.c +++ b/fs/bcachefs/rcu_pending.c @@ -182,11 +182,6 @@ static inline void kfree_bulk(size_t nr, void ** p) while (nr--) kfree(*p); } - -#define local_irq_save(flags) \ -do { \ - flags = 0; \ -} while (0) #endif static noinline void __process_finished_items(struct rcu_pending *pending, @@ -429,9 +424,15 @@ __rcu_pending_enqueue(struct rcu_pending *pending, struct rcu_head *head, BUG_ON((ptr != NULL) != (pending->process == RCU_PENDING_KVFREE_FN)); - local_irq_save(flags); - p = this_cpu_ptr(pending->p); - spin_lock(&p->lock); + /* We could technically be scheduled before taking the lock and end up + * using a different cpu's rcu_pending_pcpu: that's ok, it needs a lock + * anyways + * + * And we have to do it this way to avoid breaking PREEMPT_RT, which + * redefines how spinlocks work: + */ + p = raw_cpu_ptr(pending->p); + spin_lock_irqsave(&p->lock, flags); rcu_gp_poll_state_t seq = __get_state_synchronize_rcu(pending->srcu); restart: if (may_sleep && @@ -520,9 +521,8 @@ check_expired: goto free_node; } - local_irq_save(flags); - p = this_cpu_ptr(pending->p); - spin_lock(&p->lock); + p = raw_cpu_ptr(pending->p); + spin_lock_irqsave(&p->lock, flags); goto restart; } diff --git a/fs/bcachefs/recovery.c b/fs/bcachefs/recovery.c index 1e68e61f08e8..0b21fa6ff062 100644 --- a/fs/bcachefs/recovery.c +++ b/fs/bcachefs/recovery.c @@ -99,9 +99,11 @@ int bch2_btree_lost_data(struct bch_fs *c, goto out; case BTREE_ID_snapshots: ret = __bch2_run_explicit_recovery_pass(c, msg, BCH_RECOVERY_PASS_reconstruct_snapshots, 0) ?: ret; + ret = __bch2_run_explicit_recovery_pass(c, msg, BCH_RECOVERY_PASS_check_topology, 0) ?: ret; ret = __bch2_run_explicit_recovery_pass(c, msg, BCH_RECOVERY_PASS_scan_for_btree_nodes, 0) ?: ret; goto out; default: + ret = __bch2_run_explicit_recovery_pass(c, msg, BCH_RECOVERY_PASS_check_topology, 0) ?: ret; ret = __bch2_run_explicit_recovery_pass(c, msg, BCH_RECOVERY_PASS_scan_for_btree_nodes, 0) ?: ret; goto out; } @@ -271,13 +273,24 @@ static int bch2_journal_replay_key(struct btree_trans *trans, goto out; struct btree_path *path = btree_iter_path(trans, &iter); - if (unlikely(!btree_path_node(path, k->level))) { + if (unlikely(!btree_path_node(path, k->level) && + !k->allocated)) { + struct bch_fs *c = trans->c; + + if (!(c->recovery.passes_complete & (BIT_ULL(BCH_RECOVERY_PASS_scan_for_btree_nodes)| + BIT_ULL(BCH_RECOVERY_PASS_check_topology)))) { + bch_err(c, "have key in journal replay for btree depth that does not exist, confused"); + ret = -EINVAL; + } +#if 0 bch2_trans_iter_exit(trans, &iter); bch2_trans_node_iter_init(trans, &iter, k->btree_id, k->k->k.p, BTREE_MAX_DEPTH, 0, iter_flags); ret = bch2_btree_iter_traverse(trans, &iter) ?: bch2_btree_increase_depth(trans, iter.path, 0) ?: -BCH_ERR_transaction_restart_nested; +#endif + k->overwritten = true; goto out; } @@ -739,9 +752,11 @@ int bch2_fs_recovery(struct bch_fs *c) ? min(c->opts.recovery_pass_last, BCH_RECOVERY_PASS_snapshots_read) : BCH_RECOVERY_PASS_snapshots_read; c->opts.nochanges = true; - c->opts.read_only = true; } + if (c->opts.nochanges) + c->opts.read_only = true; + mutex_lock(&c->sb_lock); struct bch_sb_field_ext *ext = bch2_sb_field_get(c->disk_sb.sb, ext); bool write_sb = false; @@ -1093,9 +1108,6 @@ use_clean: out: bch2_flush_fsck_errs(c); - if (!IS_ERR(clean)) - kfree(clean); - if (!ret && test_bit(BCH_FS_need_delete_dead_snapshots, &c->flags) && !c->opts.nochanges) { @@ -1104,6 +1116,9 @@ out: } bch_err_fn(c, ret); +final_out: + if (!IS_ERR(clean)) + kfree(clean); return ret; err: fsck_err: @@ -1117,7 +1132,7 @@ fsck_err: bch2_print_str(c, KERN_ERR, buf.buf); printbuf_exit(&buf); } - return ret; + goto final_out; } int bch2_fs_initialize(struct bch_fs *c) diff --git a/fs/bcachefs/recovery_passes.c b/fs/bcachefs/recovery_passes.c index 605588e33fb3..35ac0d64d73a 100644 --- a/fs/bcachefs/recovery_passes.c +++ b/fs/bcachefs/recovery_passes.c @@ -294,8 +294,13 @@ static bool recovery_pass_needs_set(struct bch_fs *c, enum bch_run_recovery_pass_flags *flags) { struct bch_fs_recovery *r = &c->recovery; - bool in_recovery = test_bit(BCH_FS_in_recovery, &c->flags); - bool persistent = !in_recovery || !(*flags & RUN_RECOVERY_PASS_nopersistent); + + /* + * Never run scan_for_btree_nodes persistently: check_topology will run + * it if required + */ + if (pass == BCH_RECOVERY_PASS_scan_for_btree_nodes) + *flags |= RUN_RECOVERY_PASS_nopersistent; if ((*flags & RUN_RECOVERY_PASS_ratelimit) && !bch2_recovery_pass_want_ratelimit(c, pass)) @@ -310,6 +315,8 @@ static bool recovery_pass_needs_set(struct bch_fs *c, * Otherwise, we run run_explicit_recovery_pass when we find damage, so * it should run again even if it's already run: */ + bool in_recovery = test_bit(BCH_FS_in_recovery, &c->flags); + bool persistent = !in_recovery || !(*flags & RUN_RECOVERY_PASS_nopersistent); if (persistent ? !(c->sb.recovery_passes_required & BIT_ULL(pass)) @@ -334,6 +341,7 @@ int __bch2_run_explicit_recovery_pass(struct bch_fs *c, struct bch_fs_recovery *r = &c->recovery; int ret = 0; + lockdep_assert_held(&c->sb_lock); bch2_printbuf_make_room(out, 1024); @@ -446,7 +454,7 @@ int bch2_require_recovery_pass(struct bch_fs *c, int bch2_run_print_explicit_recovery_pass(struct bch_fs *c, enum bch_recovery_pass pass) { - enum bch_run_recovery_pass_flags flags = RUN_RECOVERY_PASS_nopersistent; + enum bch_run_recovery_pass_flags flags = 0; if (!recovery_pass_needs_set(c, pass, &flags)) return 0; diff --git a/fs/bcachefs/sb-downgrade.c b/fs/bcachefs/sb-downgrade.c index b61f88450a6d..1506d05e0665 100644 --- a/fs/bcachefs/sb-downgrade.c +++ b/fs/bcachefs/sb-downgrade.c @@ -253,6 +253,7 @@ DOWNGRADE_TABLE() static int downgrade_table_extra(struct bch_fs *c, darray_char *table) { + unsigned dst_offset = table->nr; struct bch_sb_field_downgrade_entry *dst = (void *) &darray_top(*table); unsigned bytes = sizeof(*dst) + sizeof(dst->errors[0]) * le16_to_cpu(dst->nr_errors); int ret = 0; @@ -268,6 +269,9 @@ static int downgrade_table_extra(struct bch_fs *c, darray_char *table) if (ret) return ret; + dst = (void *) &table->data[dst_offset]; + dst->nr_errors = cpu_to_le16(nr_errors + 1); + /* open coded __set_bit_le64, as dst is packed and * dst->recovery_passes is misaligned */ unsigned b = BCH_RECOVERY_PASS_STABLE_check_allocations; @@ -278,7 +282,6 @@ static int downgrade_table_extra(struct bch_fs *c, darray_char *table) break; } - dst->nr_errors = cpu_to_le16(nr_errors); return ret; } diff --git a/fs/bcachefs/sb-errors_format.h b/fs/bcachefs/sb-errors_format.h index 6fdbf265e4c0..d06e73884871 100644 --- a/fs/bcachefs/sb-errors_format.h +++ b/fs/bcachefs/sb-errors_format.h @@ -134,7 +134,7 @@ enum bch_fsck_flags { x(bucket_gens_to_invalid_buckets, 121, FSCK_AUTOFIX) \ x(bucket_gens_nonzero_for_invalid_buckets, 122, FSCK_AUTOFIX) \ x(need_discard_freespace_key_to_invalid_dev_bucket, 123, 0) \ - x(need_discard_freespace_key_bad, 124, 0) \ + x(need_discard_freespace_key_bad, 124, FSCK_AUTOFIX) \ x(discarding_bucket_not_in_need_discard_btree, 291, 0) \ x(backpointer_bucket_offset_wrong, 125, 0) \ x(backpointer_level_bad, 294, 0) \ @@ -165,7 +165,7 @@ enum bch_fsck_flags { x(ptr_to_missing_replicas_entry, 149, FSCK_AUTOFIX) \ x(ptr_to_missing_stripe, 150, 0) \ x(ptr_to_incorrect_stripe, 151, 0) \ - x(ptr_gen_newer_than_bucket_gen, 152, 0) \ + x(ptr_gen_newer_than_bucket_gen, 152, FSCK_AUTOFIX) \ x(ptr_too_stale, 153, 0) \ x(stale_dirty_ptr, 154, FSCK_AUTOFIX) \ x(ptr_bucket_data_type_mismatch, 155, 0) \ @@ -236,7 +236,7 @@ enum bch_fsck_flags { x(inode_multiple_links_but_nlink_0, 207, FSCK_AUTOFIX) \ x(inode_wrong_backpointer, 208, FSCK_AUTOFIX) \ x(inode_wrong_nlink, 209, FSCK_AUTOFIX) \ - x(inode_has_child_snapshots_wrong, 287, 0) \ + x(inode_has_child_snapshots_wrong, 287, FSCK_AUTOFIX) \ x(inode_unreachable, 210, FSCK_AUTOFIX) \ x(inode_journal_seq_in_future, 299, FSCK_AUTOFIX) \ x(inode_i_sectors_underflow, 312, FSCK_AUTOFIX) \ @@ -279,8 +279,8 @@ enum bch_fsck_flags { x(root_dir_missing, 239, 0) \ x(root_inode_not_dir, 240, 0) \ x(dir_loop, 241, 0) \ - x(hash_table_key_duplicate, 242, 0) \ - x(hash_table_key_wrong_offset, 243, 0) \ + x(hash_table_key_duplicate, 242, FSCK_AUTOFIX) \ + x(hash_table_key_wrong_offset, 243, FSCK_AUTOFIX) \ x(unlinked_inode_not_on_deleted_list, 244, FSCK_AUTOFIX) \ x(reflink_p_front_pad_bad, 245, 0) \ x(journal_entry_dup_same_device, 246, 0) \ diff --git a/fs/bcachefs/sb-members.c b/fs/bcachefs/sb-members.c index 363eb0c6eb7c..6245e342a8a8 100644 --- a/fs/bcachefs/sb-members.c +++ b/fs/bcachefs/sb-members.c @@ -325,9 +325,17 @@ static void bch2_sb_members_v1_to_text(struct printbuf *out, struct bch_sb *sb, { struct bch_sb_field_members_v1 *mi = field_to_type(f, members_v1); struct bch_sb_field_disk_groups *gi = bch2_sb_field_get(sb, disk_groups); - unsigned i; - for (i = 0; i < sb->nr_devices; i++) + if (vstruct_end(&mi->field) <= (void *) &mi->_members[0]) { + prt_printf(out, "field ends before start of entries"); + return; + } + + unsigned nr = (vstruct_end(&mi->field) - (void *) &mi->_members[0]) / sizeof(mi->_members[0]); + if (nr != sb->nr_devices) + prt_printf(out, "nr_devices mismatch: have %i entries, should be %u", nr, sb->nr_devices); + + for (unsigned i = 0; i < min(sb->nr_devices, nr); i++) member_to_text(out, members_v1_get(mi, i), gi, sb, i); } @@ -341,9 +349,27 @@ static void bch2_sb_members_v2_to_text(struct printbuf *out, struct bch_sb *sb, { struct bch_sb_field_members_v2 *mi = field_to_type(f, members_v2); struct bch_sb_field_disk_groups *gi = bch2_sb_field_get(sb, disk_groups); - unsigned i; - for (i = 0; i < sb->nr_devices; i++) + if (vstruct_end(&mi->field) <= (void *) &mi->_members[0]) { + prt_printf(out, "field ends before start of entries"); + return; + } + + if (!le16_to_cpu(mi->member_bytes)) { + prt_printf(out, "member_bytes 0"); + return; + } + + unsigned nr = (vstruct_end(&mi->field) - (void *) &mi->_members[0]) / le16_to_cpu(mi->member_bytes); + if (nr != sb->nr_devices) + prt_printf(out, "nr_devices mismatch: have %i entries, should be %u", nr, sb->nr_devices); + + /* + * We call to_text() on superblock sections that haven't passed + * validate, so we can't trust sb->nr_devices. + */ + + for (unsigned i = 0; i < min(sb->nr_devices, nr); i++) member_to_text(out, members_v2_get(mi, i), gi, sb, i); } diff --git a/fs/bcachefs/super.c b/fs/bcachefs/super.c index 397a69da5a75..a5b97c9c5163 100644 --- a/fs/bcachefs/super.c +++ b/fs/bcachefs/super.c @@ -104,7 +104,7 @@ const char * const bch2_dev_write_refs[] = { #undef x static void __bch2_print_str(struct bch_fs *c, const char *prefix, - const char *str, bool nonblocking) + const char *str) { #ifdef __KERNEL__ struct stdio_redirect *stdio = bch2_fs_stdio_redirect(c); @@ -114,17 +114,12 @@ static void __bch2_print_str(struct bch_fs *c, const char *prefix, return; } #endif - bch2_print_string_as_lines(KERN_ERR, str, nonblocking); + bch2_print_string_as_lines(KERN_ERR, str); } void bch2_print_str(struct bch_fs *c, const char *prefix, const char *str) { - __bch2_print_str(c, prefix, str, false); -} - -void bch2_print_str_nonblocking(struct bch_fs *c, const char *prefix, const char *str) -{ - __bch2_print_str(c, prefix, str, true); + __bch2_print_str(c, prefix, str); } __printf(2, 0) @@ -1072,12 +1067,13 @@ noinline_for_stack static void print_mount_opts(struct bch_fs *c) { enum bch_opt_id i; - struct printbuf p = PRINTBUF; - bool first = true; + CLASS(printbuf, p)(); + bch2_log_msg_start(c, &p); prt_str(&p, "starting version "); bch2_version_to_text(&p, c->sb.version); + bool first = true; for (i = 0; i < bch2_opts_nr; i++) { const struct bch_option *opt = &bch2_opt_table[i]; u64 v = bch2_opt_get_by_id(&c->opts, i); @@ -1094,17 +1090,24 @@ static void print_mount_opts(struct bch_fs *c) } if (c->sb.version_incompat_allowed != c->sb.version) { - prt_printf(&p, "\n allowing incompatible features above "); + prt_printf(&p, "\nallowing incompatible features above "); bch2_version_to_text(&p, c->sb.version_incompat_allowed); } if (c->opts.verbose) { - prt_printf(&p, "\n features: "); + prt_printf(&p, "\nfeatures: "); prt_bitflags(&p, bch2_sb_features, c->sb.features); } - bch_info(c, "%s", p.buf); - printbuf_exit(&p); + if (c->sb.multi_device) { + prt_printf(&p, "\nwith devices"); + for_each_online_member(c, ca, BCH_DEV_READ_REF_bch2_online_devs) { + prt_char(&p, ' '); + prt_str(&p, ca->name); + } + } + + bch2_print_str(c, KERN_INFO, p.buf); } static bool bch2_fs_may_start(struct bch_fs *c) @@ -1995,6 +1998,22 @@ int bch2_dev_add(struct bch_fs *c, const char *path) goto err_late; } + /* + * We just changed the superblock UUID, invalidate cache and send a + * uevent to update /dev/disk/by-uuid + */ + invalidate_bdev(ca->disk_sb.bdev); + + char uuid_str[37]; + snprintf(uuid_str, sizeof(uuid_str), "UUID=%pUb", &c->sb.uuid); + + char *envp[] = { + "CHANGE=uuid", + uuid_str, + NULL, + }; + kobject_uevent_env(&ca->disk_sb.bdev->bd_device.kobj, KOBJ_CHANGE, envp); + up_write(&c->state_lock); out: printbuf_exit(&label); diff --git a/fs/bcachefs/util.c b/fs/bcachefs/util.c index dc3817f545fa..df9a6071fe18 100644 --- a/fs/bcachefs/util.c +++ b/fs/bcachefs/util.c @@ -262,8 +262,7 @@ static bool string_is_spaces(const char *str) return true; } -void bch2_print_string_as_lines(const char *prefix, const char *lines, - bool nonblocking) +void bch2_print_string_as_lines(const char *prefix, const char *lines) { bool locked = false; const char *p; @@ -273,12 +272,7 @@ void bch2_print_string_as_lines(const char *prefix, const char *lines, return; } - if (!nonblocking) { - console_lock(); - locked = true; - } else { - locked = console_trylock(); - } + locked = console_trylock(); while (*lines) { p = strchrnul(lines, '\n'); diff --git a/fs/bcachefs/util.h b/fs/bcachefs/util.h index 0a4b1d433621..6488f098d140 100644 --- a/fs/bcachefs/util.h +++ b/fs/bcachefs/util.h @@ -214,7 +214,7 @@ u64 bch2_read_flag_list(const char *, const char * const[]); void bch2_prt_u64_base2_nbits(struct printbuf *, u64, unsigned); void bch2_prt_u64_base2(struct printbuf *, u64); -void bch2_print_string_as_lines(const char *, const char *, bool); +void bch2_print_string_as_lines(const char *, const char *); typedef DARRAY(unsigned long) bch_stacktrace; int bch2_save_backtrace(bch_stacktrace *stack, struct task_struct *, unsigned, gfp_t); diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index b95c4cb21c13..60a621b00c65 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -409,6 +409,15 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) struct page **pages; size_t page_off; + /* + * FIXME: io_iter.count needs to be corrected to aligned + * length. Otherwise, iov_iter_get_pages_alloc2() operates + * with the initial unaligned length value. As a result, + * ceph_msg_data_cursor_init() triggers BUG_ON() in the case + * if msg->sparse_read_total > msg->data_length. + */ + subreq->io_iter.count = len; + err = iov_iter_get_pages_alloc2(&subreq->io_iter, &pages, len, &page_off); if (err < 0) { doutc(cl, "%llx.%llx failed to allocate pages, %d\n", diff --git a/fs/ceph/export.c b/fs/ceph/export.c index 150076ced937..b2f2af104679 100644 --- a/fs/ceph/export.c +++ b/fs/ceph/export.c @@ -33,12 +33,19 @@ struct ceph_nfs_snapfh { u32 hash; } __attribute__ ((packed)); +#define BYTES_PER_U32 (sizeof(u32)) +#define CEPH_FH_BASIC_SIZE \ + (sizeof(struct ceph_nfs_fh) / BYTES_PER_U32) +#define CEPH_FH_WITH_PARENT_SIZE \ + (sizeof(struct ceph_nfs_confh) / BYTES_PER_U32) +#define CEPH_FH_SNAPPED_INODE_SIZE \ + (sizeof(struct ceph_nfs_snapfh) / BYTES_PER_U32) + static int ceph_encode_snapfh(struct inode *inode, u32 *rawfh, int *max_len, struct inode *parent_inode) { struct ceph_client *cl = ceph_inode_to_client(inode); - static const int snap_handle_length = - sizeof(struct ceph_nfs_snapfh) >> 2; + static const int snap_handle_length = CEPH_FH_SNAPPED_INODE_SIZE; struct ceph_nfs_snapfh *sfh = (void *)rawfh; u64 snapid = ceph_snap(inode); int ret; @@ -88,10 +95,8 @@ static int ceph_encode_fh(struct inode *inode, u32 *rawfh, int *max_len, struct inode *parent_inode) { struct ceph_client *cl = ceph_inode_to_client(inode); - static const int handle_length = - sizeof(struct ceph_nfs_fh) >> 2; - static const int connected_handle_length = - sizeof(struct ceph_nfs_confh) >> 2; + static const int handle_length = CEPH_FH_BASIC_SIZE; + static const int connected_handle_length = CEPH_FH_WITH_PARENT_SIZE; int type; if (ceph_snap(inode) != CEPH_NOSNAP) @@ -308,7 +313,7 @@ static struct dentry *ceph_fh_to_dentry(struct super_block *sb, if (fh_type != FILEID_INO32_GEN && fh_type != FILEID_INO32_GEN_PARENT) return NULL; - if (fh_len < sizeof(*fh) / 4) + if (fh_len < sizeof(*fh) / BYTES_PER_U32) return NULL; doutc(fsc->client, "%llx\n", fh->ino); @@ -427,7 +432,7 @@ static struct dentry *ceph_fh_to_parent(struct super_block *sb, if (fh_type != FILEID_INO32_GEN_PARENT) return NULL; - if (fh_len < sizeof(*cfh) / 4) + if (fh_len < sizeof(*cfh) / BYTES_PER_U32) return NULL; doutc(fsc->client, "%llx\n", cfh->parent_ino); diff --git a/fs/ceph/file.c b/fs/ceph/file.c index 851d70200c6b..a7254cab44cc 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -2616,7 +2616,7 @@ static int ceph_zero_objects(struct inode *inode, loff_t offset, loff_t length) s32 stripe_unit = ci->i_layout.stripe_unit; s32 stripe_count = ci->i_layout.stripe_count; s32 object_size = ci->i_layout.object_size; - u64 object_set_size = object_size * stripe_count; + u64 object_set_size = (u64) object_size * stripe_count; u64 nearly, t; /* round offset up to next period boundary */ diff --git a/fs/ceph/super.c b/fs/ceph/super.c index f3951253e393..2b8438d8a324 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -1033,8 +1033,7 @@ void ceph_umount_begin(struct super_block *sb) struct ceph_fs_client *fsc = ceph_sb_to_fs_client(sb); doutc(fsc->client, "starting forced umount\n"); - if (!fsc) - return; + fsc->mount_state = CEPH_MOUNT_SHUTDOWN; __ceph_umount_begin(fsc); } @@ -1227,6 +1226,7 @@ static int ceph_set_super(struct super_block *s, struct fs_context *fc) s->s_time_min = 0; s->s_time_max = U32_MAX; s->s_flags |= SB_NODIRATIME | SB_NOATIME; + s->s_magic = CEPH_SUPER_MAGIC; ceph_fscrypt_set_ops(s); diff --git a/fs/dlm/lock.c b/fs/dlm/lock.c index e01d5f29f4d2..6dd3a524cd35 100644 --- a/fs/dlm/lock.c +++ b/fs/dlm/lock.c @@ -509,7 +509,7 @@ static void add_scan(struct dlm_ls *ls, struct dlm_rsb *r) void dlm_rsb_scan(struct timer_list *timer) { - struct dlm_ls *ls = from_timer(ls, timer, ls_scan_timer); + struct dlm_ls *ls = timer_container_of(ls, timer, ls_scan_timer); int our_nodeid = dlm_our_nodeid(); struct dlm_rsb *r; int rv; diff --git a/fs/ext4/super.c b/fs/ext4/super.c index a7f80ca01174..c7d39da7e733 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -3627,7 +3627,7 @@ int ext4_feature_set_ok(struct super_block *sb, int readonly) */ static void print_daily_error_info(struct timer_list *t) { - struct ext4_sb_info *sbi = from_timer(sbi, t, s_err_report); + struct ext4_sb_info *sbi = timer_container_of(sbi, t, s_err_report); struct super_block *sb = sbi->s_sb; struct ext4_super_block *es = sbi->s_es; diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c index 6d5e76848733..d480b94117cd 100644 --- a/fs/jbd2/journal.c +++ b/fs/jbd2/journal.c @@ -134,7 +134,7 @@ static __be32 jbd2_superblock_csum(journal_superblock_t *sb) static void commit_timeout(struct timer_list *t) { - journal_t *journal = from_timer(journal, t, j_commit_timer); + journal_t *journal = timer_container_of(journal, t, j_commit_timer); wake_up_process(journal->j_task); } diff --git a/fs/jffs2/erase.c b/fs/jffs2/erase.c index ef3a1e1b6cb0..fda9f4d6093f 100644 --- a/fs/jffs2/erase.c +++ b/fs/jffs2/erase.c @@ -425,7 +425,9 @@ static void jffs2_mark_erased_block(struct jffs2_sb_info *c, struct jffs2_eraseb .totlen = cpu_to_je32(c->cleanmarker_size) }; - jffs2_prealloc_raw_node_refs(c, jeb, 1); + ret = jffs2_prealloc_raw_node_refs(c, jeb, 1); + if (ret) + goto filebad; marker.hdr_crc = cpu_to_je32(crc32(0, &marker, sizeof(struct jffs2_unknown_node)-4)); diff --git a/fs/jffs2/scan.c b/fs/jffs2/scan.c index 29671e33a171..62879c218d4b 100644 --- a/fs/jffs2/scan.c +++ b/fs/jffs2/scan.c @@ -256,7 +256,9 @@ int jffs2_scan_medium(struct jffs2_sb_info *c) jffs2_dbg(1, "%s(): Skipping %d bytes in nextblock to ensure page alignment\n", __func__, skip); - jffs2_prealloc_raw_node_refs(c, c->nextblock, 1); + ret = jffs2_prealloc_raw_node_refs(c, c->nextblock, 1); + if (ret) + goto out; jffs2_scan_dirty_space(c, c->nextblock, skip); } #endif diff --git a/fs/jffs2/summary.c b/fs/jffs2/summary.c index 4fe64519870f..d83372d3e1a0 100644 --- a/fs/jffs2/summary.c +++ b/fs/jffs2/summary.c @@ -858,7 +858,10 @@ int jffs2_sum_write_sumnode(struct jffs2_sb_info *c) spin_unlock(&c->erase_completion_lock); jeb = c->nextblock; - jffs2_prealloc_raw_node_refs(c, jeb, 1); + ret = jffs2_prealloc_raw_node_refs(c, jeb, 1); + + if (ret) + goto out; if (!c->summary->sum_num || !c->summary->sum_list_head) { JFFS2_WARNING("Empty summary info!!!\n"); @@ -872,6 +875,8 @@ int jffs2_sum_write_sumnode(struct jffs2_sb_info *c) datasize += padsize; ret = jffs2_sum_write_data(c, jeb, infosize, datasize, padsize); + +out: spin_lock(&c->erase_completion_lock); return ret; } diff --git a/fs/namespace.c b/fs/namespace.c index 2f2e93927f46..e13d9ab4f564 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -2410,7 +2410,7 @@ void drop_collected_mounts(struct vfsmount *mnt) namespace_unlock(); } -bool has_locked_children(struct mount *mnt, struct dentry *dentry) +static bool __has_locked_children(struct mount *mnt, struct dentry *dentry) { struct mount *child; @@ -2424,6 +2424,16 @@ bool has_locked_children(struct mount *mnt, struct dentry *dentry) return false; } +bool has_locked_children(struct mount *mnt, struct dentry *dentry) +{ + bool res; + + read_seqlock_excl(&mount_lock); + res = __has_locked_children(mnt, dentry); + read_sequnlock_excl(&mount_lock); + return res; +} + /* * Check that there aren't references to earlier/same mount namespaces in the * specified subtree. Such references can act as pins for mount namespaces @@ -2468,23 +2478,27 @@ struct vfsmount *clone_private_mount(const struct path *path) if (IS_MNT_UNBINDABLE(old_mnt)) return ERR_PTR(-EINVAL); - if (mnt_has_parent(old_mnt)) { - if (!check_mnt(old_mnt)) - return ERR_PTR(-EINVAL); - } else { - if (!is_mounted(&old_mnt->mnt)) - return ERR_PTR(-EINVAL); - - /* Make sure this isn't something purely kernel internal. */ - if (!is_anon_ns(old_mnt->mnt_ns)) + /* + * Make sure the source mount is acceptable. + * Anything mounted in our mount namespace is allowed. + * Otherwise, it must be the root of an anonymous mount + * namespace, and we need to make sure no namespace + * loops get created. + */ + if (!check_mnt(old_mnt)) { + if (!is_mounted(&old_mnt->mnt) || + !is_anon_ns(old_mnt->mnt_ns) || + mnt_has_parent(old_mnt)) return ERR_PTR(-EINVAL); - /* Make sure we don't create mount namespace loops. */ if (!check_for_nsfs_mounts(old_mnt)) return ERR_PTR(-EINVAL); } - if (has_locked_children(old_mnt, path->dentry)) + if (!ns_capable(old_mnt->mnt_ns->user_ns, CAP_SYS_ADMIN)) + return ERR_PTR(-EPERM); + + if (__has_locked_children(old_mnt, path->dentry)) return ERR_PTR(-EINVAL); new_mnt = clone_mnt(old_mnt, path->dentry, CL_PRIVATE); @@ -2930,6 +2944,10 @@ static int do_change_type(struct path *path, int ms_flags) return -EINVAL; namespace_lock(); + if (!check_mnt(mnt)) { + err = -EINVAL; + goto out_unlock; + } if (type == MS_SHARED) { err = invent_group_ids(mnt, recurse); if (err) @@ -3021,7 +3039,7 @@ static struct mount *__do_loopback(struct path *old_path, int recurse) if (!may_copy_tree(old_path)) return mnt; - if (!recurse && has_locked_children(old, old_path->dentry)) + if (!recurse && __has_locked_children(old, old_path->dentry)) return mnt; if (recurse) @@ -3414,7 +3432,7 @@ static int do_set_group(struct path *from_path, struct path *to_path) goto out; /* From mount should not have locked children in place of To's root */ - if (has_locked_children(from, to->mnt.mnt_root)) + if (__has_locked_children(from, to->mnt.mnt_root)) goto out; /* Setting sharing groups is only allowed on private mounts */ @@ -3428,7 +3446,7 @@ static int do_set_group(struct path *from_path, struct path *to_path) if (IS_MNT_SLAVE(from)) { struct mount *m = from->mnt_master; - list_add(&to->mnt_slave, &m->mnt_slave_list); + list_add(&to->mnt_slave, &from->mnt_slave); to->mnt_master = m; } @@ -3453,18 +3471,25 @@ out: * Check if path is overmounted, i.e., if there's a mount on top of * @path->mnt with @path->dentry as mountpoint. * - * Context: This function expects namespace_lock() to be held. + * Context: namespace_sem must be held at least shared. + * MUST NOT be called under lock_mount_hash() (there one should just + * call __lookup_mnt() and check if it returns NULL). * Return: If path is overmounted true is returned, false if not. */ static inline bool path_overmounted(const struct path *path) { + unsigned seq = read_seqbegin(&mount_lock); + bool no_child; + rcu_read_lock(); - if (unlikely(__lookup_mnt(path->mnt, path->dentry))) { - rcu_read_unlock(); - return true; - } + no_child = !__lookup_mnt(path->mnt, path->dentry); rcu_read_unlock(); - return false; + if (need_seqretry(&mount_lock, seq)) { + read_seqlock_excl(&mount_lock); + no_child = !__lookup_mnt(path->mnt, path->dentry); + read_sequnlock_excl(&mount_lock); + } + return unlikely(!no_child); } /** @@ -3623,37 +3648,41 @@ static int do_move_mount(struct path *old_path, ns = old->mnt_ns; err = -EINVAL; - if (!may_use_mount(p)) - goto out; - /* The thing moved must be mounted... */ if (!is_mounted(&old->mnt)) goto out; - /* ... and either ours or the root of anon namespace */ - if (!(attached ? check_mnt(old) : is_anon_ns(ns))) - goto out; - - if (is_anon_ns(ns) && ns == p->mnt_ns) { + if (check_mnt(old)) { + /* if the source is in our namespace... */ + /* ... it should be detachable from parent */ + if (!mnt_has_parent(old) || IS_MNT_LOCKED(old)) + goto out; + /* ... and the target should be in our namespace */ + if (!check_mnt(p)) + goto out; + } else { /* - * Ending up with two files referring to the root of the - * same anonymous mount namespace would cause an error - * as this would mean trying to move the same mount - * twice into the mount tree which would be rejected - * later. But be explicit about it right here. + * otherwise the source must be the root of some anon namespace. + * AV: check for mount being root of an anon namespace is worth + * an inlined predicate... */ - goto out; - } else if (is_anon_ns(p->mnt_ns)) { + if (!is_anon_ns(ns) || mnt_has_parent(old)) + goto out; /* - * Don't allow moving an attached mount tree to an - * anonymous mount tree. + * Bail out early if the target is within the same namespace - + * subsequent checks would've rejected that, but they lose + * some corner cases if we check it early. */ - goto out; + if (ns == p->mnt_ns) + goto out; + /* + * Target should be either in our namespace or in an acceptable + * anon namespace, sensu check_anonymous_mnt(). + */ + if (!may_use_mount(p)) + goto out; } - if (old->mnt.mnt_flags & MNT_LOCKED) - goto out; - if (!path_mounted(old_path)) goto out; diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c index 61a4141f8d6b..f15ca6fc400d 100644 --- a/fs/nilfs2/segment.c +++ b/fs/nilfs2/segment.c @@ -2485,7 +2485,7 @@ static int nilfs_segctor_construct(struct nilfs_sc_info *sci, int mode) static void nilfs_construction_timeout(struct timer_list *t) { - struct nilfs_sc_info *sci = from_timer(sci, t, sc_timer); + struct nilfs_sc_info *sci = timer_container_of(sci, t, sc_timer); wake_up_process(sci->sc_task); } diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c index 43e652a2adaf..b05d4e9d13b2 100644 --- a/fs/ocfs2/cluster/tcp.c +++ b/fs/ocfs2/cluster/tcp.c @@ -1488,7 +1488,8 @@ static void o2net_sc_send_keep_req(struct work_struct *work) * where shutdown is going to be involved */ static void o2net_idle_timer(struct timer_list *t) { - struct o2net_sock_container *sc = from_timer(sc, t, sc_idle_timeout); + struct o2net_sock_container *sc = timer_container_of(sc, t, + sc_idle_timeout); struct o2net_node *nn = o2net_nn_from_num(sc->sc_node->nd_num); #ifdef CONFIG_DEBUG_FS unsigned long msecs = ktime_to_ms(ktime_get()) - diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 969b458100fe..dfea7bd800cb 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -48,8 +48,8 @@ static struct file *ovl_open_realfile(const struct file *file, if (!inode_owner_or_capable(real_idmap, realinode)) flags &= ~O_NOATIME; - realfile = backing_file_open(&file->f_path, flags, realpath, - current_cred()); + realfile = backing_file_open(file_user_path((struct file *) file), + flags, realpath, current_cred()); } ovl_revert_creds(old_cred); diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c index bf722daf19a9..0b8b28392eb7 100644 --- a/fs/overlayfs/namei.c +++ b/fs/overlayfs/namei.c @@ -16,6 +16,7 @@ struct ovl_lookup_data { struct super_block *sb; + struct dentry *dentry; const struct ovl_layer *layer; struct qstr name; bool is_dir; @@ -24,6 +25,7 @@ struct ovl_lookup_data { bool stop; bool last; char *redirect; + char *upperredirect; int metacopy; /* Referring to last redirect xattr */ bool absolute_redirect; @@ -1024,6 +1026,31 @@ int ovl_verify_lowerdata(struct dentry *dentry) return ovl_maybe_validate_verity(dentry); } +/* + * Following redirects/metacopy can have security consequences: it's like a + * symlink into the lower layer without the permission checks. + * + * This is only a problem if the upper layer is untrusted (e.g comes from an USB + * drive). This can allow a non-readable file or directory to become readable. + * + * Only following redirects when redirects are enabled disables this attack + * vector when not necessary. + */ +static bool ovl_check_follow_redirect(struct ovl_lookup_data *d) +{ + struct ovl_fs *ofs = OVL_FS(d->sb); + + if (d->metacopy && !ofs->config.metacopy) { + pr_warn_ratelimited("refusing to follow metacopy origin for (%pd2)\n", d->dentry); + return false; + } + if ((d->redirect || d->upperredirect) && !ovl_redirect_follow(ofs)) { + pr_warn_ratelimited("refusing to follow redirect for (%pd2)\n", d->dentry); + return false; + } + return true; +} + struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags) { @@ -1039,7 +1066,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, unsigned int ctr = 0; struct inode *inode = NULL; bool upperopaque = false; - char *upperredirect = NULL; + bool check_redirect = (ovl_redirect_follow(ofs) || ofs->numdatalayer); struct dentry *this; unsigned int i; int err; @@ -1047,12 +1074,14 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, int metacopy_size = 0; struct ovl_lookup_data d = { .sb = dentry->d_sb, + .dentry = dentry, .name = dentry->d_name, .is_dir = false, .opaque = false, .stop = false, - .last = ovl_redirect_follow(ofs) ? false : !ovl_numlower(poe), + .last = check_redirect ? false : !ovl_numlower(poe), .redirect = NULL, + .upperredirect = NULL, .metacopy = 0, }; @@ -1094,8 +1123,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, if (d.redirect) { err = -ENOMEM; - upperredirect = kstrdup(d.redirect, GFP_KERNEL); - if (!upperredirect) + d.upperredirect = kstrdup(d.redirect, GFP_KERNEL); + if (!d.upperredirect) goto out_put_upper; if (d.redirect[0] == '/') poe = roe; @@ -1113,7 +1142,12 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, for (i = 0; !d.stop && i < ovl_numlower(poe); i++) { struct ovl_path lower = ovl_lowerstack(poe)[i]; - if (!ovl_redirect_follow(ofs)) + if (!ovl_check_follow_redirect(&d)) { + err = -EPERM; + goto out_put; + } + + if (!check_redirect) d.last = i == ovl_numlower(poe) - 1; else if (d.is_dir || !ofs->numdatalayer) d.last = lower.layer->idx == ovl_numlower(roe); @@ -1126,13 +1160,6 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, if (!this) continue; - if ((uppermetacopy || d.metacopy) && !ofs->config.metacopy) { - dput(this); - err = -EPERM; - pr_warn_ratelimited("refusing to follow metacopy origin for (%pd2)\n", dentry); - goto out_put; - } - /* * If no origin fh is stored in upper of a merge dir, store fh * of lower dir and set upper parent "impure". @@ -1185,23 +1212,6 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, ctr++; } - /* - * Following redirects can have security consequences: it's like - * a symlink into the lower layer without the permission checks. - * This is only a problem if the upper layer is untrusted (e.g - * comes from an USB drive). This can allow a non-readable file - * or directory to become readable. - * - * Only following redirects when redirects are enabled disables - * this attack vector when not necessary. - */ - err = -EPERM; - if (d.redirect && !ovl_redirect_follow(ofs)) { - pr_warn_ratelimited("refusing to follow redirect for (%pd2)\n", - dentry); - goto out_put; - } - if (d.stop) break; @@ -1212,10 +1222,16 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, } } - /* Defer lookup of lowerdata in data-only layers to first access */ + /* + * Defer lookup of lowerdata in data-only layers to first access. + * Don't require redirect=follow and metacopy=on in this case. + */ if (d.metacopy && ctr && ofs->numdatalayer && d.absolute_redirect) { d.metacopy = 0; ctr++; + } else if (!ovl_check_follow_redirect(&d)) { + err = -EPERM; + goto out_put; } /* @@ -1298,20 +1314,26 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, /* * It's safe to assign upperredirect here: the previous - * assignment of happens only if upperdentry is non-NULL, and + * assignment happens only if upperdentry is non-NULL, and * this one only if upperdentry is NULL. */ - upperredirect = ovl_get_redirect_xattr(ofs, &upperpath, 0); - if (IS_ERR(upperredirect)) { - err = PTR_ERR(upperredirect); - upperredirect = NULL; + d.upperredirect = ovl_get_redirect_xattr(ofs, &upperpath, 0); + if (IS_ERR(d.upperredirect)) { + err = PTR_ERR(d.upperredirect); + d.upperredirect = NULL; goto out_free_oe; } + err = ovl_check_metacopy_xattr(ofs, &upperpath, NULL); if (err < 0) goto out_free_oe; - uppermetacopy = err; + d.metacopy = uppermetacopy = err; metacopy_size = err; + + if (!ovl_check_follow_redirect(&d)) { + err = -EPERM; + goto out_free_oe; + } } if (upperdentry || ctr) { @@ -1319,7 +1341,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, .upperdentry = upperdentry, .oe = oe, .index = index, - .redirect = upperredirect, + .redirect = d.upperredirect, }; /* Store lowerdata redirect for lazy lookup */ @@ -1361,7 +1383,7 @@ out_put_upper: kfree(origin_path); } dput(upperdentry); - kfree(upperredirect); + kfree(d.upperredirect); out: kfree(d.redirect); ovl_revert_creds(old_cred); diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h index cb449ab310a7..afb7762f873f 100644 --- a/fs/overlayfs/ovl_entry.h +++ b/fs/overlayfs/ovl_entry.h @@ -51,7 +51,7 @@ struct ovl_path { struct ovl_entry { unsigned int __numlower; - struct ovl_path __lowerstack[]; + struct ovl_path __lowerstack[] __counted_by(__numlower); }; /* private information held for overlayfs's superblock */ diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c index 6759f7d040c8..f42488c01957 100644 --- a/fs/overlayfs/params.c +++ b/fs/overlayfs/params.c @@ -871,18 +871,6 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, config->uuid = OVL_UUID_NULL; } - /* Resolve verity -> metacopy dependency */ - if (config->verity_mode && !config->metacopy) { - /* Don't allow explicit specified conflicting combinations */ - if (set.metacopy) { - pr_err("conflicting options: metacopy=off,verity=%s\n", - ovl_verity_mode(config)); - return -EINVAL; - } - /* Otherwise automatically enable metacopy. */ - config->metacopy = true; - } - /* * This is to make the logic below simpler. It doesn't make any other * difference, since redirect_dir=on is only used for upper. @@ -890,18 +878,13 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, if (!config->upperdir && config->redirect_mode == OVL_REDIRECT_FOLLOW) config->redirect_mode = OVL_REDIRECT_ON; - /* Resolve verity -> metacopy -> redirect_dir dependency */ + /* metacopy -> redirect_dir dependency */ if (config->metacopy && config->redirect_mode != OVL_REDIRECT_ON) { if (set.metacopy && set.redirect) { pr_err("conflicting options: metacopy=on,redirect_dir=%s\n", ovl_redirect_mode(config)); return -EINVAL; } - if (config->verity_mode && set.redirect) { - pr_err("conflicting options: verity=%s,redirect_dir=%s\n", - ovl_verity_mode(config), ovl_redirect_mode(config)); - return -EINVAL; - } if (set.redirect) { /* * There was an explicit redirect_dir=... that resulted @@ -970,7 +953,7 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, } - /* Resolve userxattr -> !redirect && !metacopy && !verity dependency */ + /* Resolve userxattr -> !redirect && !metacopy dependency */ if (config->userxattr) { if (set.redirect && config->redirect_mode != OVL_REDIRECT_NOFOLLOW) { @@ -982,11 +965,6 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, pr_err("conflicting options: userxattr,metacopy=on\n"); return -EINVAL; } - if (config->verity_mode) { - pr_err("conflicting options: userxattr,verity=%s\n", - ovl_verity_mode(config)); - return -EINVAL; - } /* * Silently disable default setting of redirect and metacopy. * This shall be the default in the future as well: these @@ -1025,11 +1003,6 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, */ } - if (ctx->nr_data > 0 && !config->metacopy) { - pr_err("lower data-only dirs require metacopy support.\n"); - return -EINVAL; - } - return 0; } @@ -1078,17 +1051,16 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry) seq_printf(m, ",redirect_dir=%s", ovl_redirect_mode(&ofs->config)); if (ofs->config.index != ovl_index_def) - seq_printf(m, ",index=%s", ofs->config.index ? "on" : "off"); + seq_printf(m, ",index=%s", str_on_off(ofs->config.index)); if (ofs->config.uuid != ovl_uuid_def()) seq_printf(m, ",uuid=%s", ovl_uuid_mode(&ofs->config)); if (ofs->config.nfs_export != ovl_nfs_export_def) - seq_printf(m, ",nfs_export=%s", ofs->config.nfs_export ? - "on" : "off"); + seq_printf(m, ",nfs_export=%s", + str_on_off(ofs->config.nfs_export)); if (ofs->config.xino != ovl_xino_def() && !ovl_same_fs(ofs)) seq_printf(m, ",xino=%s", ovl_xino_mode(&ofs->config)); if (ofs->config.metacopy != ovl_metacopy_def) - seq_printf(m, ",metacopy=%s", - ofs->config.metacopy ? "on" : "off"); + seq_printf(m, ",metacopy=%s", str_on_off(ofs->config.metacopy)); if (ofs->config.ovl_volatile) seq_puts(m, ",volatile"); if (ofs->config.userxattr) diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 44e208da417c..474c80d210d1 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -13,6 +13,7 @@ #include <linux/security.h> #include <linux/cred.h> #include <linux/ratelimit.h> +#include <linux/overflow.h> #include "overlayfs.h" struct ovl_cache_entry { @@ -147,9 +148,8 @@ static struct ovl_cache_entry *ovl_cache_entry_new(struct ovl_readdir_data *rdd, u64 ino, unsigned int d_type) { struct ovl_cache_entry *p; - size_t size = offsetof(struct ovl_cache_entry, name[len + 1]); - p = kmalloc(size, GFP_KERNEL); + p = kmalloc(struct_size(p, name, len + 1), GFP_KERNEL); if (!p) return NULL; diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c index 0819c739cc2f..dcccb4b4a66c 100644 --- a/fs/overlayfs/util.c +++ b/fs/overlayfs/util.c @@ -15,6 +15,7 @@ #include <linux/uuid.h> #include <linux/namei.h> #include <linux/ratelimit.h> +#include <linux/overflow.h> #include "overlayfs.h" /* Get write access to upper mnt - may fail if upper sb was remounted ro */ @@ -145,9 +146,9 @@ void ovl_stack_free(struct ovl_path *stack, unsigned int n) struct ovl_entry *ovl_alloc_entry(unsigned int numlower) { - size_t size = offsetof(struct ovl_entry, __lowerstack[numlower]); - struct ovl_entry *oe = kzalloc(size, GFP_KERNEL); + struct ovl_entry *oe; + oe = kzalloc(struct_size(oe, __lowerstack, numlower), GFP_KERNEL); if (oe) oe->__numlower = numlower; @@ -305,7 +306,9 @@ enum ovl_path_type ovl_path_realdata(struct dentry *dentry, struct path *path) struct dentry *ovl_dentry_upper(struct dentry *dentry) { - return ovl_upperdentry_dereference(OVL_I(d_inode(dentry))); + struct inode *inode = d_inode(dentry); + + return inode ? ovl_upperdentry_dereference(OVL_I(inode)) : NULL; } struct dentry *ovl_dentry_lower(struct dentry *dentry) diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c index cc37f58b47dd..1beb124e25f6 100644 --- a/fs/resctrl/rdtgroup.c +++ b/fs/resctrl/rdtgroup.c @@ -536,6 +536,8 @@ static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of, goto unlock; } + rdt_last_cmd_clear(); + if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED || rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) { ret = -EINVAL; @@ -3472,6 +3474,8 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_unlock; } + rdt_last_cmd_clear(); + /* * Check that the parent directory for a monitor group is a "mon_groups" * directory. diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c index e03c890de0a0..c0196be0e65f 100644 --- a/fs/smb/client/cifs_debug.c +++ b/fs/smb/client/cifs_debug.c @@ -362,6 +362,10 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v) c = 0; spin_lock(&cifs_tcp_ses_lock); list_for_each_entry(server, &cifs_tcp_ses_list, tcp_ses_list) { +#ifdef CONFIG_CIFS_SMB_DIRECT + struct smbdirect_socket_parameters *sp; +#endif + /* channel info will be printed as a part of sessions below */ if (SERVER_IS_CHAN(server)) continue; @@ -383,25 +387,26 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v) seq_printf(m, "\nSMBDirect transport not available"); goto skip_rdma; } + sp = &server->smbd_conn->socket.parameters; seq_printf(m, "\nSMBDirect (in hex) protocol version: %x " "transport status: %x", server->smbd_conn->protocol, - server->smbd_conn->transport_status); + server->smbd_conn->socket.status); seq_printf(m, "\nConn receive_credit_max: %x " "send_credit_target: %x max_send_size: %x", - server->smbd_conn->receive_credit_max, - server->smbd_conn->send_credit_target, - server->smbd_conn->max_send_size); + sp->recv_credit_max, + sp->send_credit_target, + sp->max_send_size); seq_printf(m, "\nConn max_fragmented_recv_size: %x " "max_fragmented_send_size: %x max_receive_size:%x", - server->smbd_conn->max_fragmented_recv_size, - server->smbd_conn->max_fragmented_send_size, - server->smbd_conn->max_receive_size); + sp->max_fragmented_recv_size, + sp->max_fragmented_send_size, + sp->max_recv_size); seq_printf(m, "\nConn keep_alive_interval: %x " "max_readwrite_size: %x rdma_readwrite_threshold: %x", - server->smbd_conn->keep_alive_interval, - server->smbd_conn->max_readwrite_size, + sp->keepalive_interval_msec * 1000, + sp->max_read_write_size, server->smbd_conn->rdma_readwrite_threshold); seq_printf(m, "\nDebug count_get_receive_buffer: %x " "count_put_receive_buffer: %x count_send_empty: %x", diff --git a/fs/smb/client/cifsfs.h b/fs/smb/client/cifsfs.h index ca435a3841b8..b9ec9fe16a98 100644 --- a/fs/smb/client/cifsfs.h +++ b/fs/smb/client/cifsfs.h @@ -145,6 +145,6 @@ extern const struct export_operations cifs_export_ops; #endif /* CONFIG_CIFS_NFSD_EXPORT */ /* when changing internal version - update following two lines at same time */ -#define SMB3_PRODUCT_BUILD 54 -#define CIFS_VERSION "2.54" +#define SMB3_PRODUCT_BUILD 55 +#define CIFS_VERSION "2.55" #endif /* _CIFSFS_H */ diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h index ad7dd16db3e9..45e94e18f4d5 100644 --- a/fs/smb/client/cifsglob.h +++ b/fs/smb/client/cifsglob.h @@ -1085,6 +1085,7 @@ struct cifs_chan { }; #define CIFS_SES_FLAG_SCALE_CHANNELS (0x1) +#define CIFS_SES_FLAGS_PENDING_QUERY_INTERFACES (0x2) /* * Session structure. One of these for each uid session with a particular host diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c index 024817d40c5f..28bc33496623 100644 --- a/fs/smb/client/connect.c +++ b/fs/smb/client/connect.c @@ -116,13 +116,9 @@ static void smb2_query_server_interfaces(struct work_struct *work) rc = server->ops->query_server_interfaces(xid, tcon, false); free_xid(xid); - if (rc) { - if (rc == -EOPNOTSUPP) - return; - + if (rc) cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n", __func__, rc); - } queue_delayed_work(cifsiod_wq, &tcon->query_interfaces, (SMB_INTERFACE_POLL_INTERVAL * HZ)); diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c index bab9f567d9b7..1468c16ea9b8 100644 --- a/fs/smb/client/smb2ops.c +++ b/fs/smb/client/smb2ops.c @@ -504,6 +504,9 @@ smb3_negotiate_wsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx) wsize = min_t(unsigned int, wsize, server->max_write); #ifdef CONFIG_CIFS_SMB_DIRECT if (server->rdma) { + struct smbdirect_socket_parameters *sp = + &server->smbd_conn->socket.parameters; + if (server->sign) /* * Account for SMB2 data transfer packet header and @@ -511,12 +514,12 @@ smb3_negotiate_wsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx) */ wsize = min_t(unsigned int, wsize, - server->smbd_conn->max_fragmented_send_size - + sp->max_fragmented_send_size - SMB2_READWRITE_PDU_HEADER_SIZE - sizeof(struct smb2_transform_hdr)); else wsize = min_t(unsigned int, - wsize, server->smbd_conn->max_readwrite_size); + wsize, sp->max_read_write_size); } #endif if (!(server->capabilities & SMB2_GLOBAL_CAP_LARGE_MTU)) @@ -552,6 +555,9 @@ smb3_negotiate_rsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx) rsize = min_t(unsigned int, rsize, server->max_read); #ifdef CONFIG_CIFS_SMB_DIRECT if (server->rdma) { + struct smbdirect_socket_parameters *sp = + &server->smbd_conn->socket.parameters; + if (server->sign) /* * Account for SMB2 data transfer packet header and @@ -559,12 +565,12 @@ smb3_negotiate_rsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx) */ rsize = min_t(unsigned int, rsize, - server->smbd_conn->max_fragmented_recv_size - + sp->max_fragmented_recv_size - SMB2_READWRITE_PDU_HEADER_SIZE - sizeof(struct smb2_transform_hdr)); else rsize = min_t(unsigned int, - rsize, server->smbd_conn->max_readwrite_size); + rsize, sp->max_read_write_size); } #endif diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c index 0c320d06809c..a717be1626a3 100644 --- a/fs/smb/client/smb2pdu.c +++ b/fs/smb/client/smb2pdu.c @@ -36,6 +36,7 @@ #include "smb2glob.h" #include "cifspdu.h" #include "cifs_spnego.h" +#include "../common/smbdirect/smbdirect.h" #include "smbdirect.h" #include "trace.h" #ifdef CONFIG_CIFS_DFS_UPCALL @@ -411,14 +412,23 @@ skip_sess_setup: if (!rc && (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL) && server->ops->query_server_interfaces) { - mutex_unlock(&ses->session_mutex); - /* - * query server network interfaces, in case they change + * query server network interfaces, in case they change. + * Also mark the session as pending this update while the query + * is in progress. This will be used to avoid calling + * smb2_reconnect recursively. */ + ses->flags |= CIFS_SES_FLAGS_PENDING_QUERY_INTERFACES; xid = get_xid(); rc = server->ops->query_server_interfaces(xid, tcon, false); free_xid(xid); + ses->flags &= ~CIFS_SES_FLAGS_PENDING_QUERY_INTERFACES; + + /* regardless of rc value, setup polling */ + queue_delayed_work(cifsiod_wq, &tcon->query_interfaces, + (SMB_INTERFACE_POLL_INTERVAL * HZ)); + + mutex_unlock(&ses->session_mutex); if (rc == -EOPNOTSUPP && ses->chan_count > 1) { /* @@ -438,11 +448,8 @@ skip_sess_setup: if (ses->chan_max > ses->chan_count && ses->iface_count && !SERVER_IS_CHAN(server)) { - if (ses->chan_count == 1) { + if (ses->chan_count == 1) cifs_server_dbg(VFS, "supports multichannel now\n"); - queue_delayed_work(cifsiod_wq, &tcon->query_interfaces, - (SMB_INTERFACE_POLL_INTERVAL * HZ)); - } cifs_try_adding_channels(ses); } @@ -560,11 +567,18 @@ static int smb2_ioctl_req_init(u32 opcode, struct cifs_tcon *tcon, struct TCP_Server_Info *server, void **request_buf, unsigned int *total_len) { - /* Skip reconnect only for FSCTL_VALIDATE_NEGOTIATE_INFO IOCTLs */ - if (opcode == FSCTL_VALIDATE_NEGOTIATE_INFO) { + /* + * Skip reconnect in one of the following cases: + * 1. For FSCTL_VALIDATE_NEGOTIATE_INFO IOCTLs + * 2. For FSCTL_QUERY_NETWORK_INTERFACE_INFO IOCTL when called from + * smb2_reconnect (indicated by CIFS_SES_FLAG_SCALE_CHANNELS ses flag) + */ + if (opcode == FSCTL_VALIDATE_NEGOTIATE_INFO || + (opcode == FSCTL_QUERY_NETWORK_INTERFACE_INFO && + (tcon->ses->flags & CIFS_SES_FLAGS_PENDING_QUERY_INTERFACES))) return __smb2_plain_req_init(SMB2_IOCTL, tcon, server, request_buf, total_len); - } + return smb2_plain_req_init(SMB2_IOCTL, tcon, server, request_buf, total_len); } @@ -4449,10 +4463,10 @@ smb2_new_read_req(void **buf, unsigned int *total_len, #ifdef CONFIG_CIFS_SMB_DIRECT /* * If we want to do a RDMA write, fill in and append - * smbd_buffer_descriptor_v1 to the end of read request + * smbdirect_buffer_descriptor_v1 to the end of read request */ if (rdata && smb3_use_rdma_offload(io_parms)) { - struct smbd_buffer_descriptor_v1 *v1; + struct smbdirect_buffer_descriptor_v1 *v1; bool need_invalidate = server->dialect == SMB30_PROT_ID; rdata->mr = smbd_register_mr(server->smbd_conn, &rdata->subreq.io_iter, @@ -4466,8 +4480,8 @@ smb2_new_read_req(void **buf, unsigned int *total_len, req->ReadChannelInfoOffset = cpu_to_le16(offsetof(struct smb2_read_req, Buffer)); req->ReadChannelInfoLength = - cpu_to_le16(sizeof(struct smbd_buffer_descriptor_v1)); - v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0]; + cpu_to_le16(sizeof(struct smbdirect_buffer_descriptor_v1)); + v1 = (struct smbdirect_buffer_descriptor_v1 *) &req->Buffer[0]; v1->offset = cpu_to_le64(rdata->mr->mr->iova); v1->token = cpu_to_le32(rdata->mr->mr->rkey); v1->length = cpu_to_le32(rdata->mr->mr->length); @@ -4975,10 +4989,10 @@ smb2_async_writev(struct cifs_io_subrequest *wdata) #ifdef CONFIG_CIFS_SMB_DIRECT /* * If we want to do a server RDMA read, fill in and append - * smbd_buffer_descriptor_v1 to the end of write request + * smbdirect_buffer_descriptor_v1 to the end of write request */ if (smb3_use_rdma_offload(io_parms)) { - struct smbd_buffer_descriptor_v1 *v1; + struct smbdirect_buffer_descriptor_v1 *v1; bool need_invalidate = server->dialect == SMB30_PROT_ID; wdata->mr = smbd_register_mr(server->smbd_conn, &wdata->subreq.io_iter, @@ -4997,8 +5011,8 @@ smb2_async_writev(struct cifs_io_subrequest *wdata) req->WriteChannelInfoOffset = cpu_to_le16(offsetof(struct smb2_write_req, Buffer)); req->WriteChannelInfoLength = - cpu_to_le16(sizeof(struct smbd_buffer_descriptor_v1)); - v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0]; + cpu_to_le16(sizeof(struct smbdirect_buffer_descriptor_v1)); + v1 = (struct smbdirect_buffer_descriptor_v1 *) &req->Buffer[0]; v1->offset = cpu_to_le64(wdata->mr->mr->iova); v1->token = cpu_to_le32(wdata->mr->mr->rkey); v1->length = cpu_to_le32(wdata->mr->mr->length); diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c index b0b7254661e9..5ae847919da5 100644 --- a/fs/smb/client/smbdirect.c +++ b/fs/smb/client/smbdirect.c @@ -7,6 +7,7 @@ #include <linux/module.h> #include <linux/highmem.h> #include <linux/folio_queue.h> +#include "../common/smbdirect/smbdirect_pdu.h" #include "smbdirect.h" #include "cifs_debug.h" #include "cifsproto.h" @@ -50,9 +51,6 @@ struct smb_extract_to_rdma { static ssize_t smb_extract_iter_to_rdma(struct iov_iter *iter, size_t len, struct smb_extract_to_rdma *rdma); -/* SMBD version number */ -#define SMBD_V1 0x0100 - /* Port numbers for SMBD transport */ #define SMB_PORT 445 #define SMBD_PORT 5445 @@ -165,10 +163,11 @@ static void smbd_disconnect_rdma_work(struct work_struct *work) { struct smbd_connection *info = container_of(work, struct smbd_connection, disconnect_work); + struct smbdirect_socket *sc = &info->socket; - if (info->transport_status == SMBD_CONNECTED) { - info->transport_status = SMBD_DISCONNECTING; - rdma_disconnect(info->id); + if (sc->status == SMBDIRECT_SOCKET_CONNECTED) { + sc->status = SMBDIRECT_SOCKET_DISCONNECTING; + rdma_disconnect(sc->rdma.cm_id); } } @@ -182,6 +181,7 @@ static int smbd_conn_upcall( struct rdma_cm_id *id, struct rdma_cm_event *event) { struct smbd_connection *info = id->context; + struct smbdirect_socket *sc = &info->socket; log_rdma_event(INFO, "event=%d status=%d\n", event->event, event->status); @@ -205,7 +205,7 @@ static int smbd_conn_upcall( case RDMA_CM_EVENT_ESTABLISHED: log_rdma_event(INFO, "connected event=%d\n", event->event); - info->transport_status = SMBD_CONNECTED; + sc->status = SMBDIRECT_SOCKET_CONNECTED; wake_up_interruptible(&info->conn_wait); break; @@ -213,20 +213,20 @@ static int smbd_conn_upcall( case RDMA_CM_EVENT_UNREACHABLE: case RDMA_CM_EVENT_REJECTED: log_rdma_event(INFO, "connecting failed event=%d\n", event->event); - info->transport_status = SMBD_DISCONNECTED; + sc->status = SMBDIRECT_SOCKET_DISCONNECTED; wake_up_interruptible(&info->conn_wait); break; case RDMA_CM_EVENT_DEVICE_REMOVAL: case RDMA_CM_EVENT_DISCONNECTED: /* This happens when we fail the negotiation */ - if (info->transport_status == SMBD_NEGOTIATE_FAILED) { - info->transport_status = SMBD_DISCONNECTED; + if (sc->status == SMBDIRECT_SOCKET_NEGOTIATE_FAILED) { + sc->status = SMBDIRECT_SOCKET_DISCONNECTED; wake_up(&info->conn_wait); break; } - info->transport_status = SMBD_DISCONNECTED; + sc->status = SMBDIRECT_SOCKET_DISCONNECTED; wake_up_interruptible(&info->disconn_wait); wake_up_interruptible(&info->wait_reassembly_queue); wake_up_interruptible_all(&info->wait_send_queue); @@ -275,6 +275,8 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc) int i; struct smbd_request *request = container_of(wc->wr_cqe, struct smbd_request, cqe); + struct smbd_connection *info = request->info; + struct smbdirect_socket *sc = &info->socket; log_rdma_send(INFO, "smbd_request 0x%p completed wc->status=%d\n", request, wc->status); @@ -286,7 +288,7 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc) } for (i = 0; i < request->num_sge; i++) - ib_dma_unmap_single(request->info->id->device, + ib_dma_unmap_single(sc->ib.dev, request->sge[i].addr, request->sge[i].length, DMA_TO_DEVICE); @@ -299,7 +301,7 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc) mempool_free(request, request->info->request_mempool); } -static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp) +static void dump_smbdirect_negotiate_resp(struct smbdirect_negotiate_resp *resp) { log_rdma_event(INFO, "resp message min_version %u max_version %u negotiated_version %u credits_requested %u credits_granted %u status %u max_readwrite_size %u preferred_send_size %u max_receive_size %u max_fragmented_size %u\n", resp->min_version, resp->max_version, @@ -318,15 +320,17 @@ static bool process_negotiation_response( struct smbd_response *response, int packet_length) { struct smbd_connection *info = response->info; - struct smbd_negotiate_resp *packet = smbd_response_payload(response); + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; + struct smbdirect_negotiate_resp *packet = smbd_response_payload(response); - if (packet_length < sizeof(struct smbd_negotiate_resp)) { + if (packet_length < sizeof(struct smbdirect_negotiate_resp)) { log_rdma_event(ERR, "error: packet_length=%d\n", packet_length); return false; } - if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) { + if (le16_to_cpu(packet->negotiated_version) != SMBDIRECT_V1) { log_rdma_event(ERR, "error: negotiated_version=%x\n", le16_to_cpu(packet->negotiated_version)); return false; @@ -347,20 +351,20 @@ static bool process_negotiation_response( atomic_set(&info->receive_credits, 0); - if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) { + if (le32_to_cpu(packet->preferred_send_size) > sp->max_recv_size) { log_rdma_event(ERR, "error: preferred_send_size=%d\n", le32_to_cpu(packet->preferred_send_size)); return false; } - info->max_receive_size = le32_to_cpu(packet->preferred_send_size); + sp->max_recv_size = le32_to_cpu(packet->preferred_send_size); if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) { log_rdma_event(ERR, "error: max_receive_size=%d\n", le32_to_cpu(packet->max_receive_size)); return false; } - info->max_send_size = min_t(int, info->max_send_size, - le32_to_cpu(packet->max_receive_size)); + sp->max_send_size = min_t(u32, sp->max_send_size, + le32_to_cpu(packet->max_receive_size)); if (le32_to_cpu(packet->max_fragmented_size) < SMBD_MIN_FRAGMENTED_SIZE) { @@ -368,18 +372,18 @@ static bool process_negotiation_response( le32_to_cpu(packet->max_fragmented_size)); return false; } - info->max_fragmented_send_size = + sp->max_fragmented_send_size = le32_to_cpu(packet->max_fragmented_size); info->rdma_readwrite_threshold = - rdma_readwrite_threshold > info->max_fragmented_send_size ? - info->max_fragmented_send_size : + rdma_readwrite_threshold > sp->max_fragmented_send_size ? + sp->max_fragmented_send_size : rdma_readwrite_threshold; - info->max_readwrite_size = min_t(u32, + sp->max_read_write_size = min_t(u32, le32_to_cpu(packet->max_readwrite_size), info->max_frmr_depth * PAGE_SIZE); - info->max_frmr_depth = info->max_readwrite_size / PAGE_SIZE; + info->max_frmr_depth = sp->max_read_write_size / PAGE_SIZE; return true; } @@ -393,8 +397,9 @@ static void smbd_post_send_credits(struct work_struct *work) struct smbd_connection *info = container_of(work, struct smbd_connection, post_send_credits_work); + struct smbdirect_socket *sc = &info->socket; - if (info->transport_status != SMBD_CONNECTED) { + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { wake_up(&info->wait_receive_queues); return; } @@ -448,7 +453,7 @@ static void smbd_post_send_credits(struct work_struct *work) /* Called from softirq, when recv is done */ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) { - struct smbd_data_transfer *data_transfer; + struct smbdirect_data_transfer *data_transfer; struct smbd_response *response = container_of(wc->wr_cqe, struct smbd_response, cqe); struct smbd_connection *info = response->info; @@ -474,7 +479,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) switch (response->type) { /* SMBD negotiation response */ case SMBD_NEGOTIATE_RESP: - dump_smbd_negotiate_resp(smbd_response_payload(response)); + dump_smbdirect_negotiate_resp(smbd_response_payload(response)); info->full_packet_received = true; info->negotiate_done = process_negotiation_response(response, wc->byte_len); @@ -531,7 +536,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) /* Send a KEEP_ALIVE response right away if requested */ info->keep_alive_requested = KEEP_ALIVE_NONE; if (le16_to_cpu(data_transfer->flags) & - SMB_DIRECT_RESPONSE_REQUESTED) { + SMBDIRECT_FLAG_RESPONSE_REQUESTED) { info->keep_alive_requested = KEEP_ALIVE_PENDING; } @@ -635,32 +640,34 @@ static int smbd_ia_open( struct smbd_connection *info, struct sockaddr *dstaddr, int port) { + struct smbdirect_socket *sc = &info->socket; int rc; - info->id = smbd_create_id(info, dstaddr, port); - if (IS_ERR(info->id)) { - rc = PTR_ERR(info->id); + sc->rdma.cm_id = smbd_create_id(info, dstaddr, port); + if (IS_ERR(sc->rdma.cm_id)) { + rc = PTR_ERR(sc->rdma.cm_id); goto out1; } + sc->ib.dev = sc->rdma.cm_id->device; - if (!frwr_is_supported(&info->id->device->attrs)) { + if (!frwr_is_supported(&sc->ib.dev->attrs)) { log_rdma_event(ERR, "Fast Registration Work Requests (FRWR) is not supported\n"); log_rdma_event(ERR, "Device capability flags = %llx max_fast_reg_page_list_len = %u\n", - info->id->device->attrs.device_cap_flags, - info->id->device->attrs.max_fast_reg_page_list_len); + sc->ib.dev->attrs.device_cap_flags, + sc->ib.dev->attrs.max_fast_reg_page_list_len); rc = -EPROTONOSUPPORT; goto out2; } info->max_frmr_depth = min_t(int, smbd_max_frmr_depth, - info->id->device->attrs.max_fast_reg_page_list_len); + sc->ib.dev->attrs.max_fast_reg_page_list_len); info->mr_type = IB_MR_TYPE_MEM_REG; - if (info->id->device->attrs.kernel_cap_flags & IBK_SG_GAPS_REG) + if (sc->ib.dev->attrs.kernel_cap_flags & IBK_SG_GAPS_REG) info->mr_type = IB_MR_TYPE_SG_GAPS; - info->pd = ib_alloc_pd(info->id->device, 0); - if (IS_ERR(info->pd)) { - rc = PTR_ERR(info->pd); + sc->ib.pd = ib_alloc_pd(sc->ib.dev, 0); + if (IS_ERR(sc->ib.pd)) { + rc = PTR_ERR(sc->ib.pd); log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc); goto out2; } @@ -668,8 +675,8 @@ static int smbd_ia_open( return 0; out2: - rdma_destroy_id(info->id); - info->id = NULL; + rdma_destroy_id(sc->rdma.cm_id); + sc->rdma.cm_id = NULL; out1: return rc; @@ -683,10 +690,12 @@ out1: */ static int smbd_post_send_negotiate_req(struct smbd_connection *info) { + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; struct ib_send_wr send_wr; int rc = -ENOMEM; struct smbd_request *request; - struct smbd_negotiate_req *packet; + struct smbdirect_negotiate_req *packet; request = mempool_alloc(info->request_mempool, GFP_KERNEL); if (!request) @@ -695,29 +704,29 @@ static int smbd_post_send_negotiate_req(struct smbd_connection *info) request->info = info; packet = smbd_request_payload(request); - packet->min_version = cpu_to_le16(SMBD_V1); - packet->max_version = cpu_to_le16(SMBD_V1); + packet->min_version = cpu_to_le16(SMBDIRECT_V1); + packet->max_version = cpu_to_le16(SMBDIRECT_V1); packet->reserved = 0; - packet->credits_requested = cpu_to_le16(info->send_credit_target); - packet->preferred_send_size = cpu_to_le32(info->max_send_size); - packet->max_receive_size = cpu_to_le32(info->max_receive_size); + packet->credits_requested = cpu_to_le16(sp->send_credit_target); + packet->preferred_send_size = cpu_to_le32(sp->max_send_size); + packet->max_receive_size = cpu_to_le32(sp->max_recv_size); packet->max_fragmented_size = - cpu_to_le32(info->max_fragmented_recv_size); + cpu_to_le32(sp->max_fragmented_recv_size); request->num_sge = 1; request->sge[0].addr = ib_dma_map_single( - info->id->device, (void *)packet, + sc->ib.dev, (void *)packet, sizeof(*packet), DMA_TO_DEVICE); - if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) { + if (ib_dma_mapping_error(sc->ib.dev, request->sge[0].addr)) { rc = -EIO; goto dma_mapping_failed; } request->sge[0].length = sizeof(*packet); - request->sge[0].lkey = info->pd->local_dma_lkey; + request->sge[0].lkey = sc->ib.pd->local_dma_lkey; ib_dma_sync_single_for_device( - info->id->device, request->sge[0].addr, + sc->ib.dev, request->sge[0].addr, request->sge[0].length, DMA_TO_DEVICE); request->cqe.done = send_done; @@ -734,14 +743,14 @@ static int smbd_post_send_negotiate_req(struct smbd_connection *info) request->sge[0].length, request->sge[0].lkey); atomic_inc(&info->send_pending); - rc = ib_post_send(info->id->qp, &send_wr, NULL); + rc = ib_post_send(sc->ib.qp, &send_wr, NULL); if (!rc) return 0; /* if we reach here, post send failed */ log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc); atomic_dec(&info->send_pending); - ib_dma_unmap_single(info->id->device, request->sge[0].addr, + ib_dma_unmap_single(sc->ib.dev, request->sge[0].addr, request->sge[0].length, DMA_TO_DEVICE); smbd_disconnect_rdma_connection(info); @@ -774,10 +783,10 @@ static int manage_credits_prior_sending(struct smbd_connection *info) /* * Check if we need to send a KEEP_ALIVE message * The idle connection timer triggers a KEEP_ALIVE message when expires - * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send + * SMBDIRECT_FLAG_RESPONSE_REQUESTED is set in the message flag to have peer send * back a response. * return value: - * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set + * 1 if SMBDIRECT_FLAG_RESPONSE_REQUESTED needs to be set * 0: otherwise */ static int manage_keep_alive_before_sending(struct smbd_connection *info) @@ -793,6 +802,8 @@ static int manage_keep_alive_before_sending(struct smbd_connection *info) static int smbd_post_send(struct smbd_connection *info, struct smbd_request *request) { + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; struct ib_send_wr send_wr; int rc, i; @@ -801,7 +812,7 @@ static int smbd_post_send(struct smbd_connection *info, "rdma_request sge[%d] addr=0x%llx length=%u\n", i, request->sge[i].addr, request->sge[i].length); ib_dma_sync_single_for_device( - info->id->device, + sc->ib.dev, request->sge[i].addr, request->sge[i].length, DMA_TO_DEVICE); @@ -816,7 +827,7 @@ static int smbd_post_send(struct smbd_connection *info, send_wr.opcode = IB_WR_SEND; send_wr.send_flags = IB_SEND_SIGNALED; - rc = ib_post_send(info->id->qp, &send_wr, NULL); + rc = ib_post_send(sc->ib.qp, &send_wr, NULL); if (rc) { log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc); smbd_disconnect_rdma_connection(info); @@ -824,7 +835,7 @@ static int smbd_post_send(struct smbd_connection *info, } else /* Reset timer for idle connection after packet is sent */ mod_delayed_work(info->workqueue, &info->idle_timer_work, - info->keep_alive_interval*HZ); + msecs_to_jiffies(sp->keepalive_interval_msec)); return rc; } @@ -833,22 +844,24 @@ static int smbd_post_send_iter(struct smbd_connection *info, struct iov_iter *iter, int *_remaining_data_length) { + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; int i, rc; int header_length; int data_length; struct smbd_request *request; - struct smbd_data_transfer *packet; + struct smbdirect_data_transfer *packet; int new_credits = 0; wait_credit: /* Wait for send credits. A SMBD packet needs one credit */ rc = wait_event_interruptible(info->wait_send_queue, atomic_read(&info->send_credits) > 0 || - info->transport_status != SMBD_CONNECTED); + sc->status != SMBDIRECT_SOCKET_CONNECTED); if (rc) goto err_wait_credit; - if (info->transport_status != SMBD_CONNECTED) { + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { log_outgoing(ERR, "disconnected not sending on wait_credit\n"); rc = -EAGAIN; goto err_wait_credit; @@ -860,17 +873,17 @@ wait_credit: wait_send_queue: wait_event(info->wait_post_send, - atomic_read(&info->send_pending) < info->send_credit_target || - info->transport_status != SMBD_CONNECTED); + atomic_read(&info->send_pending) < sp->send_credit_target || + sc->status != SMBDIRECT_SOCKET_CONNECTED); - if (info->transport_status != SMBD_CONNECTED) { + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { log_outgoing(ERR, "disconnected not sending on wait_send_queue\n"); rc = -EAGAIN; goto err_wait_send_queue; } if (unlikely(atomic_inc_return(&info->send_pending) > - info->send_credit_target)) { + sp->send_credit_target)) { atomic_dec(&info->send_pending); goto wait_send_queue; } @@ -890,8 +903,8 @@ wait_send_queue: .nr_sge = 1, .max_sge = SMBDIRECT_MAX_SEND_SGE, .sge = request->sge, - .device = info->id->device, - .local_dma_lkey = info->pd->local_dma_lkey, + .device = sc->ib.dev, + .local_dma_lkey = sc->ib.pd->local_dma_lkey, .direction = DMA_TO_DEVICE, }; @@ -909,7 +922,7 @@ wait_send_queue: /* Fill in the packet header */ packet = smbd_request_payload(request); - packet->credits_requested = cpu_to_le16(info->send_credit_target); + packet->credits_requested = cpu_to_le16(sp->send_credit_target); new_credits = manage_credits_prior_sending(info); atomic_add(new_credits, &info->receive_credits); @@ -919,7 +932,7 @@ wait_send_queue: packet->flags = 0; if (manage_keep_alive_before_sending(info)) - packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED); + packet->flags |= cpu_to_le16(SMBDIRECT_FLAG_RESPONSE_REQUESTED); packet->reserved = 0; if (!data_length) @@ -938,23 +951,23 @@ wait_send_queue: le32_to_cpu(packet->remaining_data_length)); /* Map the packet to DMA */ - header_length = sizeof(struct smbd_data_transfer); + header_length = sizeof(struct smbdirect_data_transfer); /* If this is a packet without payload, don't send padding */ if (!data_length) - header_length = offsetof(struct smbd_data_transfer, padding); + header_length = offsetof(struct smbdirect_data_transfer, padding); - request->sge[0].addr = ib_dma_map_single(info->id->device, + request->sge[0].addr = ib_dma_map_single(sc->ib.dev, (void *)packet, header_length, DMA_TO_DEVICE); - if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) { + if (ib_dma_mapping_error(sc->ib.dev, request->sge[0].addr)) { rc = -EIO; request->sge[0].addr = 0; goto err_dma; } request->sge[0].length = header_length; - request->sge[0].lkey = info->pd->local_dma_lkey; + request->sge[0].lkey = sc->ib.pd->local_dma_lkey; rc = smbd_post_send(info, request); if (!rc) @@ -963,7 +976,7 @@ wait_send_queue: err_dma: for (i = 0; i < request->num_sge; i++) if (request->sge[i].addr) - ib_dma_unmap_single(info->id->device, + ib_dma_unmap_single(sc->ib.dev, request->sge[i].addr, request->sge[i].length, DMA_TO_DEVICE); @@ -1008,17 +1021,19 @@ static int smbd_post_send_empty(struct smbd_connection *info) static int smbd_post_recv( struct smbd_connection *info, struct smbd_response *response) { + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; struct ib_recv_wr recv_wr; int rc = -EIO; response->sge.addr = ib_dma_map_single( - info->id->device, response->packet, - info->max_receive_size, DMA_FROM_DEVICE); - if (ib_dma_mapping_error(info->id->device, response->sge.addr)) + sc->ib.dev, response->packet, + sp->max_recv_size, DMA_FROM_DEVICE); + if (ib_dma_mapping_error(sc->ib.dev, response->sge.addr)) return rc; - response->sge.length = info->max_receive_size; - response->sge.lkey = info->pd->local_dma_lkey; + response->sge.length = sp->max_recv_size; + response->sge.lkey = sc->ib.pd->local_dma_lkey; response->cqe.done = recv_done; @@ -1027,9 +1042,9 @@ static int smbd_post_recv( recv_wr.sg_list = &response->sge; recv_wr.num_sge = 1; - rc = ib_post_recv(info->id->qp, &recv_wr, NULL); + rc = ib_post_recv(sc->ib.qp, &recv_wr, NULL); if (rc) { - ib_dma_unmap_single(info->id->device, response->sge.addr, + ib_dma_unmap_single(sc->ib.dev, response->sge.addr, response->sge.length, DMA_FROM_DEVICE); smbd_disconnect_rdma_connection(info); log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc); @@ -1187,9 +1202,10 @@ static struct smbd_response *get_receive_buffer(struct smbd_connection *info) static void put_receive_buffer( struct smbd_connection *info, struct smbd_response *response) { + struct smbdirect_socket *sc = &info->socket; unsigned long flags; - ib_dma_unmap_single(info->id->device, response->sge.addr, + ib_dma_unmap_single(sc->ib.dev, response->sge.addr, response->sge.length, DMA_FROM_DEVICE); spin_lock_irqsave(&info->receive_queue_lock, flags); @@ -1264,6 +1280,8 @@ static void idle_connection_timer(struct work_struct *work) struct smbd_connection *info = container_of( work, struct smbd_connection, idle_timer_work.work); + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; if (info->keep_alive_requested != KEEP_ALIVE_NONE) { log_keep_alive(ERR, @@ -1278,7 +1296,7 @@ static void idle_connection_timer(struct work_struct *work) /* Setup the next idle timeout work */ queue_delayed_work(info->workqueue, &info->idle_timer_work, - info->keep_alive_interval*HZ); + msecs_to_jiffies(sp->keepalive_interval_msec)); } /* @@ -1289,6 +1307,8 @@ static void idle_connection_timer(struct work_struct *work) void smbd_destroy(struct TCP_Server_Info *server) { struct smbd_connection *info = server->smbd_conn; + struct smbdirect_socket *sc; + struct smbdirect_socket_parameters *sp; struct smbd_response *response; unsigned long flags; @@ -1296,19 +1316,22 @@ void smbd_destroy(struct TCP_Server_Info *server) log_rdma_event(INFO, "rdma session already destroyed\n"); return; } + sc = &info->socket; + sp = &sc->parameters; log_rdma_event(INFO, "destroying rdma session\n"); - if (info->transport_status != SMBD_DISCONNECTED) { - rdma_disconnect(server->smbd_conn->id); + if (sc->status != SMBDIRECT_SOCKET_DISCONNECTED) { + rdma_disconnect(sc->rdma.cm_id); log_rdma_event(INFO, "wait for transport being disconnected\n"); wait_event_interruptible( info->disconn_wait, - info->transport_status == SMBD_DISCONNECTED); + sc->status == SMBDIRECT_SOCKET_DISCONNECTED); } log_rdma_event(INFO, "destroying qp\n"); - ib_drain_qp(info->id->qp); - rdma_destroy_qp(info->id); + ib_drain_qp(sc->ib.qp); + rdma_destroy_qp(sc->rdma.cm_id); + sc->ib.qp = NULL; log_rdma_event(INFO, "cancelling idle timer\n"); cancel_delayed_work_sync(&info->idle_timer_work); @@ -1336,7 +1359,7 @@ void smbd_destroy(struct TCP_Server_Info *server) log_rdma_event(INFO, "free receive buffers\n"); wait_event(info->wait_receive_queues, info->count_receive_queue + info->count_empty_packet_queue - == info->receive_credit_max); + == sp->recv_credit_max); destroy_receive_buffers(info); /* @@ -1355,10 +1378,10 @@ void smbd_destroy(struct TCP_Server_Info *server) } destroy_mr_list(info); - ib_free_cq(info->send_cq); - ib_free_cq(info->recv_cq); - ib_dealloc_pd(info->pd); - rdma_destroy_id(info->id); + ib_free_cq(sc->ib.send_cq); + ib_free_cq(sc->ib.recv_cq); + ib_dealloc_pd(sc->ib.pd); + rdma_destroy_id(sc->rdma.cm_id); /* free mempools */ mempool_destroy(info->request_mempool); @@ -1367,7 +1390,7 @@ void smbd_destroy(struct TCP_Server_Info *server) mempool_destroy(info->response_mempool); kmem_cache_destroy(info->response_cache); - info->transport_status = SMBD_DESTROYED; + sc->status = SMBDIRECT_SOCKET_DESTROYED; destroy_workqueue(info->workqueue); log_rdma_event(INFO, "rdma session destroyed\n"); @@ -1392,7 +1415,7 @@ int smbd_reconnect(struct TCP_Server_Info *server) * This is possible if transport is disconnected and we haven't received * notification from RDMA, but upper layer has detected timeout */ - if (server->smbd_conn->transport_status == SMBD_CONNECTED) { + if (server->smbd_conn->socket.status == SMBDIRECT_SOCKET_CONNECTED) { log_rdma_event(INFO, "disconnecting transport\n"); smbd_destroy(server); } @@ -1424,6 +1447,8 @@ static void destroy_caches_and_workqueue(struct smbd_connection *info) #define MAX_NAME_LEN 80 static int allocate_caches_and_workqueue(struct smbd_connection *info) { + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; char name[MAX_NAME_LEN]; int rc; @@ -1432,13 +1457,13 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info) kmem_cache_create( name, sizeof(struct smbd_request) + - sizeof(struct smbd_data_transfer), + sizeof(struct smbdirect_data_transfer), 0, SLAB_HWCACHE_ALIGN, NULL); if (!info->request_cache) return -ENOMEM; info->request_mempool = - mempool_create(info->send_credit_target, mempool_alloc_slab, + mempool_create(sp->send_credit_target, mempool_alloc_slab, mempool_free_slab, info->request_cache); if (!info->request_mempool) goto out1; @@ -1448,13 +1473,13 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info) kmem_cache_create( name, sizeof(struct smbd_response) + - info->max_receive_size, + sp->max_recv_size, 0, SLAB_HWCACHE_ALIGN, NULL); if (!info->response_cache) goto out2; info->response_mempool = - mempool_create(info->receive_credit_max, mempool_alloc_slab, + mempool_create(sp->recv_credit_max, mempool_alloc_slab, mempool_free_slab, info->response_cache); if (!info->response_mempool) goto out3; @@ -1464,7 +1489,7 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info) if (!info->workqueue) goto out4; - rc = allocate_receive_buffers(info, info->receive_credit_max); + rc = allocate_receive_buffers(info, sp->recv_credit_max); if (rc) { log_rdma_event(ERR, "failed to allocate receive buffers\n"); goto out5; @@ -1491,6 +1516,8 @@ static struct smbd_connection *_smbd_get_connection( { int rc; struct smbd_connection *info; + struct smbdirect_socket *sc; + struct smbdirect_socket_parameters *sp; struct rdma_conn_param conn_param; struct ib_qp_init_attr qp_attr; struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr; @@ -1500,101 +1527,102 @@ static struct smbd_connection *_smbd_get_connection( info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL); if (!info) return NULL; + sc = &info->socket; + sp = &sc->parameters; - info->transport_status = SMBD_CONNECTING; + sc->status = SMBDIRECT_SOCKET_CONNECTING; rc = smbd_ia_open(info, dstaddr, port); if (rc) { log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc); goto create_id_failed; } - if (smbd_send_credit_target > info->id->device->attrs.max_cqe || - smbd_send_credit_target > info->id->device->attrs.max_qp_wr) { + if (smbd_send_credit_target > sc->ib.dev->attrs.max_cqe || + smbd_send_credit_target > sc->ib.dev->attrs.max_qp_wr) { log_rdma_event(ERR, "consider lowering send_credit_target = %d. Possible CQE overrun, device reporting max_cqe %d max_qp_wr %d\n", smbd_send_credit_target, - info->id->device->attrs.max_cqe, - info->id->device->attrs.max_qp_wr); + sc->ib.dev->attrs.max_cqe, + sc->ib.dev->attrs.max_qp_wr); goto config_failed; } - if (smbd_receive_credit_max > info->id->device->attrs.max_cqe || - smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) { + if (smbd_receive_credit_max > sc->ib.dev->attrs.max_cqe || + smbd_receive_credit_max > sc->ib.dev->attrs.max_qp_wr) { log_rdma_event(ERR, "consider lowering receive_credit_max = %d. Possible CQE overrun, device reporting max_cqe %d max_qp_wr %d\n", smbd_receive_credit_max, - info->id->device->attrs.max_cqe, - info->id->device->attrs.max_qp_wr); + sc->ib.dev->attrs.max_cqe, + sc->ib.dev->attrs.max_qp_wr); goto config_failed; } - info->receive_credit_max = smbd_receive_credit_max; - info->send_credit_target = smbd_send_credit_target; - info->max_send_size = smbd_max_send_size; - info->max_fragmented_recv_size = smbd_max_fragmented_recv_size; - info->max_receive_size = smbd_max_receive_size; - info->keep_alive_interval = smbd_keep_alive_interval; + sp->recv_credit_max = smbd_receive_credit_max; + sp->send_credit_target = smbd_send_credit_target; + sp->max_send_size = smbd_max_send_size; + sp->max_fragmented_recv_size = smbd_max_fragmented_recv_size; + sp->max_recv_size = smbd_max_receive_size; + sp->keepalive_interval_msec = smbd_keep_alive_interval * 1000; - if (info->id->device->attrs.max_send_sge < SMBDIRECT_MAX_SEND_SGE || - info->id->device->attrs.max_recv_sge < SMBDIRECT_MAX_RECV_SGE) { + if (sc->ib.dev->attrs.max_send_sge < SMBDIRECT_MAX_SEND_SGE || + sc->ib.dev->attrs.max_recv_sge < SMBDIRECT_MAX_RECV_SGE) { log_rdma_event(ERR, "device %.*s max_send_sge/max_recv_sge = %d/%d too small\n", IB_DEVICE_NAME_MAX, - info->id->device->name, - info->id->device->attrs.max_send_sge, - info->id->device->attrs.max_recv_sge); + sc->ib.dev->name, + sc->ib.dev->attrs.max_send_sge, + sc->ib.dev->attrs.max_recv_sge); goto config_failed; } - info->send_cq = NULL; - info->recv_cq = NULL; - info->send_cq = - ib_alloc_cq_any(info->id->device, info, - info->send_credit_target, IB_POLL_SOFTIRQ); - if (IS_ERR(info->send_cq)) { - info->send_cq = NULL; + sc->ib.send_cq = + ib_alloc_cq_any(sc->ib.dev, info, + sp->send_credit_target, IB_POLL_SOFTIRQ); + if (IS_ERR(sc->ib.send_cq)) { + sc->ib.send_cq = NULL; goto alloc_cq_failed; } - info->recv_cq = - ib_alloc_cq_any(info->id->device, info, - info->receive_credit_max, IB_POLL_SOFTIRQ); - if (IS_ERR(info->recv_cq)) { - info->recv_cq = NULL; + sc->ib.recv_cq = + ib_alloc_cq_any(sc->ib.dev, info, + sp->recv_credit_max, IB_POLL_SOFTIRQ); + if (IS_ERR(sc->ib.recv_cq)) { + sc->ib.recv_cq = NULL; goto alloc_cq_failed; } memset(&qp_attr, 0, sizeof(qp_attr)); qp_attr.event_handler = smbd_qp_async_error_upcall; qp_attr.qp_context = info; - qp_attr.cap.max_send_wr = info->send_credit_target; - qp_attr.cap.max_recv_wr = info->receive_credit_max; + qp_attr.cap.max_send_wr = sp->send_credit_target; + qp_attr.cap.max_recv_wr = sp->recv_credit_max; qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SEND_SGE; qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_RECV_SGE; qp_attr.cap.max_inline_data = 0; qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR; qp_attr.qp_type = IB_QPT_RC; - qp_attr.send_cq = info->send_cq; - qp_attr.recv_cq = info->recv_cq; + qp_attr.send_cq = sc->ib.send_cq; + qp_attr.recv_cq = sc->ib.recv_cq; qp_attr.port_num = ~0; - rc = rdma_create_qp(info->id, info->pd, &qp_attr); + rc = rdma_create_qp(sc->rdma.cm_id, sc->ib.pd, &qp_attr); if (rc) { log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc); goto create_qp_failed; } + sc->ib.qp = sc->rdma.cm_id->qp; memset(&conn_param, 0, sizeof(conn_param)); conn_param.initiator_depth = 0; conn_param.responder_resources = - min(info->id->device->attrs.max_qp_rd_atom, + min(sc->ib.dev->attrs.max_qp_rd_atom, SMBD_CM_RESPONDER_RESOURCES); info->responder_resources = conn_param.responder_resources; log_rdma_mr(INFO, "responder_resources=%d\n", info->responder_resources); /* Need to send IRD/ORD in private data for iWARP */ - info->id->device->ops.get_port_immutable( - info->id->device, info->id->port_num, &port_immutable); + sc->ib.dev->ops.get_port_immutable( + sc->ib.dev, sc->rdma.cm_id->port_num, &port_immutable); if (port_immutable.core_cap_flags & RDMA_CORE_PORT_IWARP) { ird_ord_hdr[0] = info->responder_resources; ird_ord_hdr[1] = 1; @@ -1615,16 +1643,16 @@ static struct smbd_connection *_smbd_get_connection( init_waitqueue_head(&info->conn_wait); init_waitqueue_head(&info->disconn_wait); init_waitqueue_head(&info->wait_reassembly_queue); - rc = rdma_connect(info->id, &conn_param); + rc = rdma_connect(sc->rdma.cm_id, &conn_param); if (rc) { log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc); goto rdma_connect_failed; } wait_event_interruptible( - info->conn_wait, info->transport_status != SMBD_CONNECTING); + info->conn_wait, sc->status != SMBDIRECT_SOCKET_CONNECTING); - if (info->transport_status != SMBD_CONNECTED) { + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { log_rdma_event(ERR, "rdma_connect failed port=%d\n", port); goto rdma_connect_failed; } @@ -1640,7 +1668,7 @@ static struct smbd_connection *_smbd_get_connection( init_waitqueue_head(&info->wait_send_queue); INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer); queue_delayed_work(info->workqueue, &info->idle_timer_work, - info->keep_alive_interval*HZ); + msecs_to_jiffies(sp->keepalive_interval_msec)); init_waitqueue_head(&info->wait_send_pending); atomic_set(&info->send_pending, 0); @@ -1675,26 +1703,26 @@ allocate_mr_failed: negotiation_failed: cancel_delayed_work_sync(&info->idle_timer_work); destroy_caches_and_workqueue(info); - info->transport_status = SMBD_NEGOTIATE_FAILED; + sc->status = SMBDIRECT_SOCKET_NEGOTIATE_FAILED; init_waitqueue_head(&info->conn_wait); - rdma_disconnect(info->id); + rdma_disconnect(sc->rdma.cm_id); wait_event(info->conn_wait, - info->transport_status == SMBD_DISCONNECTED); + sc->status == SMBDIRECT_SOCKET_DISCONNECTED); allocate_cache_failed: rdma_connect_failed: - rdma_destroy_qp(info->id); + rdma_destroy_qp(sc->rdma.cm_id); create_qp_failed: alloc_cq_failed: - if (info->send_cq) - ib_free_cq(info->send_cq); - if (info->recv_cq) - ib_free_cq(info->recv_cq); + if (sc->ib.send_cq) + ib_free_cq(sc->ib.send_cq); + if (sc->ib.recv_cq) + ib_free_cq(sc->ib.recv_cq); config_failed: - ib_dealloc_pd(info->pd); - rdma_destroy_id(info->id); + ib_dealloc_pd(sc->ib.pd); + rdma_destroy_id(sc->rdma.cm_id); create_id_failed: kfree(info); @@ -1734,8 +1762,9 @@ try_again: static int smbd_recv_buf(struct smbd_connection *info, char *buf, unsigned int size) { + struct smbdirect_socket *sc = &info->socket; struct smbd_response *response; - struct smbd_data_transfer *data_transfer; + struct smbdirect_data_transfer *data_transfer; int to_copy, to_read, data_read, offset; u32 data_length, remaining_data_length, data_offset; int rc; @@ -1848,12 +1877,12 @@ read_rfc1002_done: rc = wait_event_interruptible( info->wait_reassembly_queue, info->reassembly_data_length >= size || - info->transport_status != SMBD_CONNECTED); + sc->status != SMBDIRECT_SOCKET_CONNECTED); /* Don't return any data if interrupted */ if (rc) return rc; - if (info->transport_status != SMBD_CONNECTED) { + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { log_read(ERR, "disconnected\n"); return -ECONNABORTED; } @@ -1871,6 +1900,7 @@ static int smbd_recv_page(struct smbd_connection *info, struct page *page, unsigned int page_offset, unsigned int to_read) { + struct smbdirect_socket *sc = &info->socket; int ret; char *to_address; void *page_address; @@ -1879,7 +1909,7 @@ static int smbd_recv_page(struct smbd_connection *info, ret = wait_event_interruptible( info->wait_reassembly_queue, info->reassembly_data_length >= to_read || - info->transport_status != SMBD_CONNECTED); + sc->status != SMBDIRECT_SOCKET_CONNECTED); if (ret) return ret; @@ -1954,12 +1984,14 @@ int smbd_send(struct TCP_Server_Info *server, int num_rqst, struct smb_rqst *rqst_array) { struct smbd_connection *info = server->smbd_conn; + struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket_parameters *sp = &sc->parameters; struct smb_rqst *rqst; struct iov_iter iter; unsigned int remaining_data_length, klen; int rc, i, rqst_idx; - if (info->transport_status != SMBD_CONNECTED) + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) return -EAGAIN; /* @@ -1971,10 +2003,10 @@ int smbd_send(struct TCP_Server_Info *server, for (i = 0; i < num_rqst; i++) remaining_data_length += smb_rqst_len(server, &rqst_array[i]); - if (unlikely(remaining_data_length > info->max_fragmented_send_size)) { + if (unlikely(remaining_data_length > sp->max_fragmented_send_size)) { /* assertion: payload never exceeds negotiated maximum */ log_write(ERR, "payload size %d > max size %d\n", - remaining_data_length, info->max_fragmented_send_size); + remaining_data_length, sp->max_fragmented_send_size); return -EINVAL; } @@ -2053,6 +2085,7 @@ static void smbd_mr_recovery_work(struct work_struct *work) { struct smbd_connection *info = container_of(work, struct smbd_connection, mr_recovery_work); + struct smbdirect_socket *sc = &info->socket; struct smbd_mr *smbdirect_mr; int rc; @@ -2070,7 +2103,7 @@ static void smbd_mr_recovery_work(struct work_struct *work) } smbdirect_mr->mr = ib_alloc_mr( - info->pd, info->mr_type, + sc->ib.pd, info->mr_type, info->max_frmr_depth); if (IS_ERR(smbdirect_mr->mr)) { log_rdma_mr(ERR, "ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n", @@ -2099,12 +2132,13 @@ static void smbd_mr_recovery_work(struct work_struct *work) static void destroy_mr_list(struct smbd_connection *info) { + struct smbdirect_socket *sc = &info->socket; struct smbd_mr *mr, *tmp; cancel_work_sync(&info->mr_recovery_work); list_for_each_entry_safe(mr, tmp, &info->mr_list, list) { if (mr->state == MR_INVALIDATED) - ib_dma_unmap_sg(info->id->device, mr->sgt.sgl, + ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir); ib_dereg_mr(mr->mr); kfree(mr->sgt.sgl); @@ -2121,6 +2155,7 @@ static void destroy_mr_list(struct smbd_connection *info) */ static int allocate_mr_list(struct smbd_connection *info) { + struct smbdirect_socket *sc = &info->socket; int i; struct smbd_mr *smbdirect_mr, *tmp; @@ -2136,7 +2171,7 @@ static int allocate_mr_list(struct smbd_connection *info) smbdirect_mr = kzalloc(sizeof(*smbdirect_mr), GFP_KERNEL); if (!smbdirect_mr) goto cleanup_entries; - smbdirect_mr->mr = ib_alloc_mr(info->pd, info->mr_type, + smbdirect_mr->mr = ib_alloc_mr(sc->ib.pd, info->mr_type, info->max_frmr_depth); if (IS_ERR(smbdirect_mr->mr)) { log_rdma_mr(ERR, "ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n", @@ -2181,20 +2216,20 @@ cleanup_entries: */ static struct smbd_mr *get_mr(struct smbd_connection *info) { + struct smbdirect_socket *sc = &info->socket; struct smbd_mr *ret; int rc; again: rc = wait_event_interruptible(info->wait_mr, atomic_read(&info->mr_ready_count) || - info->transport_status != SMBD_CONNECTED); + sc->status != SMBDIRECT_SOCKET_CONNECTED); if (rc) { log_rdma_mr(ERR, "wait_event_interruptible rc=%x\n", rc); return NULL; } - if (info->transport_status != SMBD_CONNECTED) { - log_rdma_mr(ERR, "info->transport_status=%x\n", - info->transport_status); + if (sc->status != SMBDIRECT_SOCKET_CONNECTED) { + log_rdma_mr(ERR, "sc->status=%x\n", sc->status); return NULL; } @@ -2247,6 +2282,7 @@ struct smbd_mr *smbd_register_mr(struct smbd_connection *info, struct iov_iter *iter, bool writing, bool need_invalidate) { + struct smbdirect_socket *sc = &info->socket; struct smbd_mr *smbdirect_mr; int rc, num_pages; enum dma_data_direction dir; @@ -2276,7 +2312,7 @@ struct smbd_mr *smbd_register_mr(struct smbd_connection *info, num_pages, iov_iter_count(iter), info->max_frmr_depth); smbd_iter_to_mr(info, iter, &smbdirect_mr->sgt, info->max_frmr_depth); - rc = ib_dma_map_sg(info->id->device, smbdirect_mr->sgt.sgl, + rc = ib_dma_map_sg(sc->ib.dev, smbdirect_mr->sgt.sgl, smbdirect_mr->sgt.nents, dir); if (!rc) { log_rdma_mr(ERR, "ib_dma_map_sg num_pages=%x dir=%x rc=%x\n", @@ -2312,7 +2348,7 @@ struct smbd_mr *smbd_register_mr(struct smbd_connection *info, * on IB_WR_REG_MR. Hardware enforces a barrier and order of execution * on the next ib_post_send when we actually send I/O to remote peer */ - rc = ib_post_send(info->id->qp, ®_wr->wr, NULL); + rc = ib_post_send(sc->ib.qp, ®_wr->wr, NULL); if (!rc) return smbdirect_mr; @@ -2321,7 +2357,7 @@ struct smbd_mr *smbd_register_mr(struct smbd_connection *info, /* If all failed, attempt to recover this MR by setting it MR_ERROR*/ map_mr_error: - ib_dma_unmap_sg(info->id->device, smbdirect_mr->sgt.sgl, + ib_dma_unmap_sg(sc->ib.dev, smbdirect_mr->sgt.sgl, smbdirect_mr->sgt.nents, smbdirect_mr->dir); dma_map_error: @@ -2359,6 +2395,7 @@ int smbd_deregister_mr(struct smbd_mr *smbdirect_mr) { struct ib_send_wr *wr; struct smbd_connection *info = smbdirect_mr->conn; + struct smbdirect_socket *sc = &info->socket; int rc = 0; if (smbdirect_mr->need_invalidate) { @@ -2372,7 +2409,7 @@ int smbd_deregister_mr(struct smbd_mr *smbdirect_mr) wr->send_flags = IB_SEND_SIGNALED; init_completion(&smbdirect_mr->invalidate_done); - rc = ib_post_send(info->id->qp, wr, NULL); + rc = ib_post_send(sc->ib.qp, wr, NULL); if (rc) { log_rdma_mr(ERR, "ib_post_send failed rc=%x\n", rc); smbd_disconnect_rdma_connection(info); @@ -2389,7 +2426,7 @@ int smbd_deregister_mr(struct smbd_mr *smbdirect_mr) if (smbdirect_mr->state == MR_INVALIDATED) { ib_dma_unmap_sg( - info->id->device, smbdirect_mr->sgt.sgl, + sc->ib.dev, smbdirect_mr->sgt.sgl, smbdirect_mr->sgt.nents, smbdirect_mr->dir); smbdirect_mr->state = MR_READY; diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h index c08e3665150d..75b3f491c3ad 100644 --- a/fs/smb/client/smbdirect.h +++ b/fs/smb/client/smbdirect.h @@ -15,6 +15,9 @@ #include <rdma/rdma_cm.h> #include <linux/mempool.h> +#include "../common/smbdirect/smbdirect.h" +#include "../common/smbdirect/smbdirect_socket.h" + extern int rdma_readwrite_threshold; extern int smbd_max_frmr_depth; extern int smbd_keep_alive_interval; @@ -50,14 +53,8 @@ enum smbd_connection_status { * 5. mempools for allocating packets */ struct smbd_connection { - enum smbd_connection_status transport_status; - - /* RDMA related */ - struct rdma_cm_id *id; - struct ib_qp_init_attr qp_attr; - struct ib_pd *pd; - struct ib_cq *send_cq, *recv_cq; - struct ib_device_attr dev_attr; + struct smbdirect_socket socket; + int ri_rc; struct completion ri_done; wait_queue_head_t conn_wait; @@ -72,15 +69,7 @@ struct smbd_connection { spinlock_t lock_new_credits_offered; int new_credits_offered; - /* Connection parameters defined in [MS-SMBD] 3.1.1.1 */ - int receive_credit_max; - int send_credit_target; - int max_send_size; - int max_fragmented_recv_size; - int max_fragmented_send_size; - int max_receive_size; - int keep_alive_interval; - int max_readwrite_size; + /* dynamic connection parameters defined in [MS-SMBD] 3.1.1.1 */ enum keep_alive_status keep_alive_requested; int protocol; atomic_t send_credits; @@ -177,54 +166,6 @@ enum smbd_message_type { SMBD_TRANSFER_DATA, }; -#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001 - -/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */ -struct smbd_negotiate_req { - __le16 min_version; - __le16 max_version; - __le16 reserved; - __le16 credits_requested; - __le32 preferred_send_size; - __le32 max_receive_size; - __le32 max_fragmented_size; -} __packed; - -/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */ -struct smbd_negotiate_resp { - __le16 min_version; - __le16 max_version; - __le16 negotiated_version; - __le16 reserved; - __le16 credits_requested; - __le16 credits_granted; - __le32 status; - __le32 max_readwrite_size; - __le32 preferred_send_size; - __le32 max_receive_size; - __le32 max_fragmented_size; -} __packed; - -/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */ -struct smbd_data_transfer { - __le16 credits_requested; - __le16 credits_granted; - __le16 flags; - __le16 reserved; - __le32 remaining_data_length; - __le32 data_offset; - __le32 data_length; - __le32 padding; - __u8 buffer[]; -} __packed; - -/* The packet fields for a registered RDMA buffer */ -struct smbd_buffer_descriptor_v1 { - __le64 offset; - __le32 token; - __le32 length; -} __packed; - /* Maximum number of SGEs used by smbdirect.c in any send work request */ #define SMBDIRECT_MAX_SEND_SGE 6 diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c index 266af17aa7d9..191783f553ce 100644 --- a/fs/smb/client/transport.c +++ b/fs/smb/client/transport.c @@ -1018,14 +1018,16 @@ struct TCP_Server_Info *cifs_pick_channel(struct cifs_ses *ses) uint index = 0; unsigned int min_in_flight = UINT_MAX, max_in_flight = 0; struct TCP_Server_Info *server = NULL; - int i; + int i, start, cur; if (!ses) return NULL; spin_lock(&ses->chan_lock); + start = atomic_inc_return(&ses->chan_seq); for (i = 0; i < ses->chan_count; i++) { - server = ses->chans[i].server; + cur = (start + i) % ses->chan_count; + server = ses->chans[cur].server; if (!server || server->terminate) continue; @@ -1042,17 +1044,15 @@ struct TCP_Server_Info *cifs_pick_channel(struct cifs_ses *ses) */ if (server->in_flight < min_in_flight) { min_in_flight = server->in_flight; - index = i; + index = cur; } if (server->in_flight > max_in_flight) max_in_flight = server->in_flight; } /* if all channels are equally loaded, fall back to round-robin */ - if (min_in_flight == max_in_flight) { - index = (uint)atomic_inc_return(&ses->chan_seq); - index %= ses->chan_count; - } + if (min_in_flight == max_in_flight) + index = (uint)start % ses->chan_count; server = ses->chans[index].server; spin_unlock(&ses->chan_lock); diff --git a/fs/smb/common/smbdirect/smbdirect.h b/fs/smb/common/smbdirect/smbdirect.h new file mode 100644 index 000000000000..b9a385344ff3 --- /dev/null +++ b/fs/smb/common/smbdirect/smbdirect.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2017, Microsoft Corporation. + * Copyright (C) 2018, LG Electronics. + */ + +#ifndef __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_H__ +#define __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_H__ + +/* SMB-DIRECT buffer descriptor V1 structure [MS-SMBD] 2.2.3.1 */ +struct smbdirect_buffer_descriptor_v1 { + __le64 offset; + __le32 token; + __le32 length; +} __packed; + +/* + * Connection parameters mostly from [MS-SMBD] 3.1.1.1 + * + * These are setup and negotiated at the beginning of a + * connection and remain constant unless explicitly changed. + * + * Some values are important for the upper layer. + */ +struct smbdirect_socket_parameters { + __u16 recv_credit_max; + __u16 send_credit_target; + __u32 max_send_size; + __u32 max_fragmented_send_size; + __u32 max_recv_size; + __u32 max_fragmented_recv_size; + __u32 max_read_write_size; + __u32 keepalive_interval_msec; + __u32 keepalive_timeout_msec; +} __packed; + +#endif /* __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_H__ */ diff --git a/fs/smb/common/smbdirect/smbdirect_pdu.h b/fs/smb/common/smbdirect/smbdirect_pdu.h new file mode 100644 index 000000000000..ae9fdb05ce23 --- /dev/null +++ b/fs/smb/common/smbdirect/smbdirect_pdu.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2017 Stefan Metzmacher + */ + +#ifndef __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_PDU_H__ +#define __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_PDU_H__ + +#define SMBDIRECT_V1 0x0100 + +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */ +struct smbdirect_negotiate_req { + __le16 min_version; + __le16 max_version; + __le16 reserved; + __le16 credits_requested; + __le32 preferred_send_size; + __le32 max_receive_size; + __le32 max_fragmented_size; +} __packed; + +/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */ +struct smbdirect_negotiate_resp { + __le16 min_version; + __le16 max_version; + __le16 negotiated_version; + __le16 reserved; + __le16 credits_requested; + __le16 credits_granted; + __le32 status; + __le32 max_readwrite_size; + __le32 preferred_send_size; + __le32 max_receive_size; + __le32 max_fragmented_size; +} __packed; + +#define SMBDIRECT_DATA_MIN_HDR_SIZE 0x14 +#define SMBDIRECT_DATA_OFFSET 0x18 + +#define SMBDIRECT_FLAG_RESPONSE_REQUESTED 0x0001 + +/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */ +struct smbdirect_data_transfer { + __le16 credits_requested; + __le16 credits_granted; + __le16 flags; + __le16 reserved; + __le32 remaining_data_length; + __le32 data_offset; + __le32 data_length; + __le32 padding; + __u8 buffer[]; +} __packed; + +#endif /* __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_PDU_H__ */ diff --git a/fs/smb/common/smbdirect/smbdirect_socket.h b/fs/smb/common/smbdirect/smbdirect_socket.h new file mode 100644 index 000000000000..e5b15cc44a7b --- /dev/null +++ b/fs/smb/common/smbdirect/smbdirect_socket.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2025 Stefan Metzmacher + */ + +#ifndef __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ +#define __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ + +enum smbdirect_socket_status { + SMBDIRECT_SOCKET_CREATED, + SMBDIRECT_SOCKET_CONNECTING, + SMBDIRECT_SOCKET_CONNECTED, + SMBDIRECT_SOCKET_NEGOTIATE_FAILED, + SMBDIRECT_SOCKET_DISCONNECTING, + SMBDIRECT_SOCKET_DISCONNECTED, + SMBDIRECT_SOCKET_DESTROYED +}; + +struct smbdirect_socket { + enum smbdirect_socket_status status; + + /* RDMA related */ + struct { + struct rdma_cm_id *cm_id; + } rdma; + + /* IB verbs related */ + struct { + struct ib_pd *pd; + struct ib_cq *send_cq; + struct ib_cq *recv_cq; + + /* + * shortcuts for rdma.cm_id->{qp,device}; + */ + struct ib_qp *qp; + struct ib_device *dev; + } ib; + + struct smbdirect_socket_parameters parameters; +}; + +#endif /* __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ */ diff --git a/fs/ubifs/journal.c b/fs/ubifs/journal.c index ee954e64ce7f..e28ab4395e5c 100644 --- a/fs/ubifs/journal.c +++ b/fs/ubifs/journal.c @@ -985,7 +985,7 @@ int ubifs_jnl_write_inode(struct ubifs_info *c, const struct inode *inode) dbg_jnl("ino %lu, nlink %u", inode->i_ino, inode->i_nlink); if (kill_xattrs && ui->xattr_cnt > ubifs_xattr_max_cnt(c)) { - ubifs_err(c, "Cannot delete inode, it has too much xattrs!"); + ubifs_err(c, "Cannot delete inode, it has too many xattrs!"); err = -EPERM; ubifs_ro_mode(c, err); return err; diff --git a/include/linux/codetag.h b/include/linux/codetag.h index 0ee4c21c6dbc..5f2b9a1f722c 100644 --- a/include/linux/codetag.h +++ b/include/linux/codetag.h @@ -36,8 +36,8 @@ union codetag_ref { struct codetag_type_desc { const char *section; size_t tag_size; - void (*module_load)(struct module *mod, - struct codetag *start, struct codetag *end); + int (*module_load)(struct module *mod, + struct codetag *start, struct codetag *end); void (*module_unload)(struct module *mod, struct codetag *start, struct codetag *end); #ifdef CONFIG_MODULES @@ -89,7 +89,7 @@ void *codetag_alloc_module_section(struct module *mod, const char *name, unsigned long align); void codetag_free_module_sections(struct module *mod); void codetag_module_replaced(struct module *mod, struct module *new_mod); -void codetag_load_module(struct module *mod); +int codetag_load_module(struct module *mod); void codetag_unload_module(struct module *mod); #else /* defined(CONFIG_CODE_TAGGING) && defined(CONFIG_MODULES) */ @@ -103,7 +103,7 @@ codetag_alloc_module_section(struct module *mod, const char *name, unsigned long align) { return NULL; } static inline void codetag_free_module_sections(struct module *mod) {} static inline void codetag_module_replaced(struct module *mod, struct module *new_mod) {} -static inline void codetag_load_module(struct module *mod) {} +static inline int codetag_load_module(struct module *mod) { return 0; } static inline void codetag_unload_module(struct module *mod) {} #endif /* defined(CONFIG_CODE_TAGGING) && defined(CONFIG_MODULES) */ diff --git a/include/linux/cpu.h b/include/linux/cpu.h index e6089abc28e2..96a3a0d6a60e 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -120,6 +120,7 @@ extern void cpu_maps_update_begin(void); extern void cpu_maps_update_done(void); int bringup_hibernate_cpu(unsigned int sleep_cpu); void bringup_nonboot_cpus(unsigned int max_cpus); +int arch_cpu_rescan_dead_smt_siblings(void); #else /* CONFIG_SMP */ #define cpuhp_tasks_frozen 0 @@ -134,6 +135,8 @@ static inline void cpu_maps_update_done(void) static inline int add_cpu(unsigned int cpu) { return 0;} +static inline int arch_cpu_rescan_dead_smt_siblings(void) { return 0; } + #endif /* CONFIG_SMP */ extern const struct bus_type cpu_subsys; diff --git a/include/linux/export.h b/include/linux/export.h index a8c23d945634..f35d03b4113b 100644 --- a/include/linux/export.h +++ b/include/linux/export.h @@ -24,11 +24,17 @@ .long sym #endif -#define ___EXPORT_SYMBOL(sym, license, ns) \ +/* + * LLVM integrated assembler cam merge adjacent string literals (like + * C and GNU-as) passed to '.ascii', but not to '.asciz' and chokes on: + * + * .asciz "MODULE_" "kvm" ; + */ +#define ___EXPORT_SYMBOL(sym, license, ns...) \ .section ".export_symbol","a" ASM_NL \ __export_symbol_##sym: ASM_NL \ .asciz license ASM_NL \ - .asciz ns ASM_NL \ + .ascii ns "\0" ASM_NL \ __EXPORT_SYMBOL_REF(sym) ASM_NL \ .previous @@ -85,4 +91,6 @@ #define EXPORT_SYMBOL_NS(sym, ns) __EXPORT_SYMBOL(sym, "", ns) #define EXPORT_SYMBOL_NS_GPL(sym, ns) __EXPORT_SYMBOL(sym, "GPL", ns) +#define EXPORT_SYMBOL_GPL_FOR_MODULES(sym, mods) __EXPORT_SYMBOL(sym, "GPL", "module:" mods) + #endif /* _LINUX_EXPORT_H */ diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a4d3816d252a..b672ca15f265 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -635,6 +635,8 @@ enum { #define ftrace_get_symaddr(fentry_ip) (0) #endif +void ftrace_sync_ipi(void *data); + #ifdef CONFIG_DYNAMIC_FTRACE void ftrace_arch_code_modify_prepare(void); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 0598f36931de..42f374e828a2 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -279,6 +279,7 @@ bool is_hugetlb_entry_migration(pte_t pte); bool is_hugetlb_entry_hwpoisoned(pte_t pte); void hugetlb_unshare_all_pmds(struct vm_area_struct *vma); void fixup_hugetlb_reservations(struct vm_area_struct *vma); +void hugetlb_split(struct vm_area_struct *vma, unsigned long addr); #else /* !CONFIG_HUGETLB_PAGE */ @@ -476,6 +477,8 @@ static inline void fixup_hugetlb_reservations(struct vm_area_struct *vma) { } +static inline void hugetlb_split(struct vm_area_struct *vma, unsigned long addr) {} + #endif /* !CONFIG_HUGETLB_PAGE */ #ifndef pgd_write diff --git a/include/linux/key.h b/include/linux/key.h index ba05de8579ec..81b8f05c6898 100644 --- a/include/linux/key.h +++ b/include/linux/key.h @@ -236,7 +236,7 @@ struct key { #define KEY_FLAG_ROOT_CAN_INVAL 7 /* set if key can be invalidated by root without permission */ #define KEY_FLAG_KEEP 8 /* set if key should not be removed */ #define KEY_FLAG_UID_KEYRING 9 /* set if key is a user or user session keyring */ -#define KEY_FLAG_FINAL_PUT 10 /* set if final put has happened on key */ +#define KEY_FLAG_USER_ALIVE 10 /* set if final put has not happened on key yet */ /* the key type and key description string * - the desc is used to match a key against search criteria diff --git a/include/linux/module.h b/include/linux/module.h index 8050f77c3b64..92e1420fccdf 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -249,8 +249,8 @@ struct module_kobject *lookup_or_create_module_kobject(const char *name); #ifdef MODULE /* Creates an alias so file2alias.c can find device table. */ #define MODULE_DEVICE_TABLE(type, name) \ -extern typeof(name) __mod_device_table__##type##__##name \ - __attribute__ ((unused, alias(__stringify(name)))) +static typeof(name) __mod_device_table__##type##__##name \ + __attribute__ ((used, alias(__stringify(name)))) #else /* !MODULE */ #define MODULE_DEVICE_TABLE(type, name) #endif diff --git a/include/linux/mount.h b/include/linux/mount.h index d3ee0e5162f0..4880f434c021 100644 --- a/include/linux/mount.h +++ b/include/linux/mount.h @@ -65,7 +65,8 @@ enum mount_flags { MNT_ATIME_MASK = MNT_NOATIME | MNT_NODIRATIME | MNT_RELATIME, MNT_INTERNAL_FLAGS = MNT_SHARED | MNT_WRITE_HOLD | MNT_INTERNAL | - MNT_DOOMED | MNT_SYNC_UMOUNT | MNT_MARKED, + MNT_DOOMED | MNT_SYNC_UMOUNT | MNT_MARKED | + MNT_LOCKED, }; struct vfsmount { diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h index 98030accf641..72ff44cca864 100644 --- a/include/linux/raid/pq.h +++ b/include/linux/raid/pq.h @@ -108,6 +108,10 @@ extern const struct raid6_calls raid6_vpermxor4; extern const struct raid6_calls raid6_vpermxor8; extern const struct raid6_calls raid6_lsx; extern const struct raid6_calls raid6_lasx; +extern const struct raid6_calls raid6_rvvx1; +extern const struct raid6_calls raid6_rvvx2; +extern const struct raid6_calls raid6_rvvx4; +extern const struct raid6_calls raid6_rvvx8; struct raid6_recov_calls { void (*data2)(int, size_t, int, int, void **); @@ -125,6 +129,7 @@ extern const struct raid6_recov_calls raid6_recov_s390xc; extern const struct raid6_recov_calls raid6_recov_neon; extern const struct raid6_recov_calls raid6_recov_lsx; extern const struct raid6_recov_calls raid6_recov_lasx; +extern const struct raid6_recov_calls raid6_recov_rvv; extern const struct raid6_calls raid6_neonx1; extern const struct raid6_calls raid6_neonx2; diff --git a/include/linux/time_namespace.h b/include/linux/time_namespace.h index 0b8b32bf0655..bb2c52f4fc94 100644 --- a/include/linux/time_namespace.h +++ b/include/linux/time_namespace.h @@ -12,6 +12,7 @@ struct user_namespace; extern struct user_namespace init_user_ns; +struct seq_file; struct vm_area_struct; struct timens_offsets { diff --git a/include/linux/timer.h b/include/linux/timer.h index f636f55c427d..0414d9e6b4fc 100644 --- a/include/linux/timer.h +++ b/include/linux/timer.h @@ -129,7 +129,7 @@ extern void timer_destroy_on_stack(struct timer_list *timer); static inline void timer_destroy_on_stack(struct timer_list *timer) { } #endif -#define from_timer(var, callback_timer, timer_fieldname) \ +#define timer_container_of(var, callback_timer, timer_fieldname) \ container_of(callback_timer, typeof(*var), timer_fieldname) /** diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h index 2b261e74e2c4..a760f05fa3fb 100644 --- a/include/net/bluetooth/hci_core.h +++ b/include/net/bluetooth/hci_core.h @@ -242,6 +242,7 @@ struct adv_info { __u8 mesh; __u8 instance; __u8 handle; + __u8 sid; __u32 flags; __u16 timeout; __u16 remaining_time; @@ -546,6 +547,7 @@ struct hci_dev { struct hci_conn_hash conn_hash; struct list_head mesh_pending; + struct mutex mgmt_pending_lock; struct list_head mgmt_pending; struct list_head reject_list; struct list_head accept_list; @@ -1550,13 +1552,14 @@ struct hci_conn *hci_connect_sco(struct hci_dev *hdev, int type, bdaddr_t *dst, u16 timeout); struct hci_conn *hci_bind_cis(struct hci_dev *hdev, bdaddr_t *dst, __u8 dst_type, struct bt_iso_qos *qos); -struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, +struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, __u8 sid, struct bt_iso_qos *qos, __u8 base_len, __u8 *base); struct hci_conn *hci_connect_cis(struct hci_dev *hdev, bdaddr_t *dst, __u8 dst_type, struct bt_iso_qos *qos); struct hci_conn *hci_connect_bis(struct hci_dev *hdev, bdaddr_t *dst, - __u8 dst_type, struct bt_iso_qos *qos, + __u8 dst_type, __u8 sid, + struct bt_iso_qos *qos, __u8 data_len, __u8 *data); struct hci_conn *hci_pa_create_sync(struct hci_dev *hdev, bdaddr_t *dst, __u8 dst_type, __u8 sid, struct bt_iso_qos *qos); @@ -1831,6 +1834,7 @@ int hci_remove_remote_oob_data(struct hci_dev *hdev, bdaddr_t *bdaddr, void hci_adv_instances_clear(struct hci_dev *hdev); struct adv_info *hci_find_adv_instance(struct hci_dev *hdev, u8 instance); +struct adv_info *hci_find_adv_sid(struct hci_dev *hdev, u8 sid); struct adv_info *hci_get_next_instance(struct hci_dev *hdev, u8 instance); struct adv_info *hci_add_adv_instance(struct hci_dev *hdev, u8 instance, u32 flags, u16 adv_data_len, u8 *adv_data, @@ -1838,7 +1842,7 @@ struct adv_info *hci_add_adv_instance(struct hci_dev *hdev, u8 instance, u16 timeout, u16 duration, s8 tx_power, u32 min_interval, u32 max_interval, u8 mesh_handle); -struct adv_info *hci_add_per_instance(struct hci_dev *hdev, u8 instance, +struct adv_info *hci_add_per_instance(struct hci_dev *hdev, u8 instance, u8 sid, u32 flags, u8 data_len, u8 *data, u32 min_interval, u32 max_interval); int hci_set_adv_instance_data(struct hci_dev *hdev, u8 instance, @@ -2400,7 +2404,6 @@ void mgmt_advertising_added(struct sock *sk, struct hci_dev *hdev, u8 instance); void mgmt_advertising_removed(struct sock *sk, struct hci_dev *hdev, u8 instance); -void mgmt_adv_monitor_removed(struct hci_dev *hdev, u16 handle); int mgmt_phy_configuration_changed(struct hci_dev *hdev, struct sock *skip); void mgmt_adv_monitor_device_lost(struct hci_dev *hdev, u16 handle, bdaddr_t *bdaddr, u8 addr_type); diff --git a/include/net/bluetooth/hci_sync.h b/include/net/bluetooth/hci_sync.h index 72558c826aa1..5224f57f6af2 100644 --- a/include/net/bluetooth/hci_sync.h +++ b/include/net/bluetooth/hci_sync.h @@ -115,8 +115,8 @@ int hci_enable_ext_advertising_sync(struct hci_dev *hdev, u8 instance); int hci_enable_advertising_sync(struct hci_dev *hdev); int hci_enable_advertising(struct hci_dev *hdev); -int hci_start_per_adv_sync(struct hci_dev *hdev, u8 instance, u8 data_len, - u8 *data, u32 flags, u16 min_interval, +int hci_start_per_adv_sync(struct hci_dev *hdev, u8 instance, u8 sid, + u8 data_len, u8 *data, u32 flags, u16 min_interval, u16 max_interval, u16 sync_interval); int hci_disable_per_advertising_sync(struct hci_dev *hdev, u8 instance); diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 629368ab2787..638948be4c50 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -973,14 +973,6 @@ static inline void qdisc_qstats_qlen_backlog(struct Qdisc *sch, __u32 *qlen, *backlog = qstats.backlog; } -static inline void qdisc_tree_flush_backlog(struct Qdisc *sch) -{ - __u32 qlen, backlog; - - qdisc_qstats_qlen_backlog(sch, &qlen, &backlog); - qdisc_tree_reduce_backlog(sch, qlen, backlog); -} - static inline void qdisc_purge_queue(struct Qdisc *sch) { __u32 qlen, backlog; diff --git a/include/net/sock.h b/include/net/sock.h index 92e7c1aae3cc..4c37015b7cf7 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -3010,8 +3010,11 @@ int sock_ioctl_inout(struct sock *sk, unsigned int cmd, int sk_ioctl(struct sock *sk, unsigned int cmd, void __user *arg); static inline bool sk_is_readable(struct sock *sk) { - if (sk->sk_prot->sock_is_readable) - return sk->sk_prot->sock_is_readable(sk); + const struct proto *prot = READ_ONCE(sk->sk_prot); + + if (prot->sock_is_readable) + return prot->sock_is_readable(sk); + return false; } #endif /* _SOCK_H */ diff --git a/include/trace/events/power.h b/include/trace/events/power.h index 9253e83b9bb4..6c631eec23e3 100644 --- a/include/trace/events/power.h +++ b/include/trace/events/power.h @@ -338,53 +338,6 @@ DEFINE_EVENT(wakeup_source, wakeup_source_deactivate, ); /* - * The clock events are used for clock enable/disable and for - * clock rate change - */ -DECLARE_EVENT_CLASS(clock, - - TP_PROTO(const char *name, unsigned int state, unsigned int cpu_id), - - TP_ARGS(name, state, cpu_id), - - TP_STRUCT__entry( - __string( name, name ) - __field( u64, state ) - __field( u64, cpu_id ) - ), - - TP_fast_assign( - __assign_str(name); - __entry->state = state; - __entry->cpu_id = cpu_id; - ), - - TP_printk("%s state=%lu cpu_id=%lu", __get_str(name), - (unsigned long)__entry->state, (unsigned long)__entry->cpu_id) -); - -DEFINE_EVENT(clock, clock_enable, - - TP_PROTO(const char *name, unsigned int state, unsigned int cpu_id), - - TP_ARGS(name, state, cpu_id) -); - -DEFINE_EVENT(clock, clock_disable, - - TP_PROTO(const char *name, unsigned int state, unsigned int cpu_id), - - TP_ARGS(name, state, cpu_id) -); - -DEFINE_EVENT(clock, clock_set_rate, - - TP_PROTO(const char *name, unsigned int state, unsigned int cpu_id), - - TP_ARGS(name, state, cpu_id) -); - -/* * The power domain events are used for power domains transitions */ DECLARE_EVENT_CLASS(power_domain, diff --git a/include/uapi/linux/bits.h b/include/uapi/linux/bits.h index 682b406e1067..a04afef9efca 100644 --- a/include/uapi/linux/bits.h +++ b/include/uapi/linux/bits.h @@ -4,9 +4,9 @@ #ifndef _UAPI_LINUX_BITS_H #define _UAPI_LINUX_BITS_H -#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (BITS_PER_LONG - 1 - (h)))) +#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (__BITS_PER_LONG - 1 - (h)))) -#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (BITS_PER_LONG_LONG - 1 - (h)))) +#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (__BITS_PER_LONG_LONG - 1 - (h)))) #define __GENMASK_U128(h, l) \ ((_BIT128((h)) << 1) - (_BIT128(l))) diff --git a/kernel/configs/tiny.config b/kernel/configs/tiny.config index b753695c5a8f..5dd0f0a34a73 100644 --- a/kernel/configs/tiny.config +++ b/kernel/configs/tiny.config @@ -2,3 +2,4 @@ CONFIG_CC_OPTIMIZE_FOR_SIZE=y CONFIG_KERNEL_XZ=y CONFIG_SLUB=y CONFIG_SLUB_TINY=y +CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y diff --git a/kernel/kthread.c b/kernel/kthread.c index 77c44924cf54..85fc068f0083 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -1207,7 +1207,8 @@ EXPORT_SYMBOL_GPL(kthread_queue_work); */ void kthread_delayed_work_timer_fn(struct timer_list *t) { - struct kthread_delayed_work *dwork = from_timer(dwork, t, timer); + struct kthread_delayed_work *dwork = timer_container_of(dwork, t, + timer); struct kthread_work *work = &dwork->work; struct kthread_worker *worker = work->worker; unsigned long flags; diff --git a/kernel/module/main.c b/kernel/module/main.c index 3d64e69cc03e..413ac6ea3702 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -170,6 +170,30 @@ static inline void add_taint_module(struct module *mod, unsigned flag, } /* + * Like strncmp(), except s/-/_/g as per scripts/Makefile.lib:name-fix-token rule. + */ +static int mod_strncmp(const char *str_a, const char *str_b, size_t n) +{ + for (int i = 0; i < n; i++) { + char a = str_a[i]; + char b = str_b[i]; + int d; + + if (a == '-') a = '_'; + if (b == '-') b = '_'; + + d = a - b; + if (d) + return d; + + if (!a) + break; + } + + return 0; +} + +/* * A thread that wants to hold a reference to a module only while it * is running can call this to safely exit. */ @@ -1083,6 +1107,46 @@ static char *get_modinfo(const struct load_info *info, const char *tag) return get_next_modinfo(info, tag, NULL); } +/** + * verify_module_namespace() - does @modname have access to this symbol's @namespace + * @namespace: export symbol namespace + * @modname: module name + * + * If @namespace is prefixed with "module:" to indicate it is a module namespace + * then test if @modname matches any of the comma separated patterns. + * + * The patterns only support tail-glob. + */ +static bool verify_module_namespace(const char *namespace, const char *modname) +{ + size_t len, modlen = strlen(modname); + const char *prefix = "module:"; + const char *sep; + bool glob; + + if (!strstarts(namespace, prefix)) + return false; + + for (namespace += strlen(prefix); *namespace; namespace = sep) { + sep = strchrnul(namespace, ','); + len = sep - namespace; + + glob = false; + if (sep[-1] == '*') { + len--; + glob = true; + } + + if (*sep) + sep++; + + if (mod_strncmp(namespace, modname, len) == 0 && (glob || len == modlen)) + return true; + } + + return false; +} + static int verify_namespace_is_imported(const struct load_info *info, const struct kernel_symbol *sym, struct module *mod) @@ -1092,6 +1156,10 @@ static int verify_namespace_is_imported(const struct load_info *info, namespace = kernel_symbol_namespace(sym); if (namespace && namespace[0]) { + + if (verify_module_namespace(namespace, mod->name)) + return 0; + for_each_modinfo_entry(imported_namespace, info, "import_ns") { if (strcmp(namespace, imported_namespace) == 0) return 0; @@ -1658,15 +1726,30 @@ static void module_license_taint_check(struct module *mod, const char *license) } } -static void setup_modinfo(struct module *mod, struct load_info *info) +static int setup_modinfo(struct module *mod, struct load_info *info) { const struct module_attribute *attr; + char *imported_namespace; int i; for (i = 0; (attr = modinfo_attrs[i]); i++) { if (attr->setup) attr->setup(mod, get_modinfo(info, attr->attr.name)); } + + for_each_modinfo_entry(imported_namespace, info, "import_ns") { + /* + * 'module:' prefixed namespaces are implicit, disallow + * explicit imports. + */ + if (strstarts(imported_namespace, "module:")) { + pr_err("%s: module tries to import module namespace: %s\n", + mod->name, imported_namespace); + return -EPERM; + } + } + + return 0; } static void free_modinfo(struct module *mod) @@ -3323,7 +3406,9 @@ static int load_module(struct load_info *info, const char __user *uargs, goto free_unload; /* Set up MODINFO_ATTR fields */ - setup_modinfo(mod, info); + err = setup_modinfo(mod, info); + if (err) + goto free_modinfo; /* Fix up syms, so that st_value is a pointer to location. */ err = simplify_symbols(mod, info); @@ -3386,11 +3471,12 @@ static int load_module(struct load_info *info, const char __user *uargs, goto sysfs_cleanup; } + if (codetag_load_module(mod)) + goto sysfs_cleanup; + /* Get rid of temporary copy. */ free_copy(info, flags); - codetag_load_module(mod); - /* Done! */ trace_module_load(mod); diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index c0cc7ae41106..f92443561d36 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -316,7 +316,8 @@ static void call_rcu_tasks_generic_timer(struct timer_list *tlp) unsigned long flags; bool needwake = false; struct rcu_tasks *rtp; - struct rcu_tasks_percpu *rtpcp = from_timer(rtpcp, tlp, lazy_timer); + struct rcu_tasks_percpu *rtpcp = timer_container_of(rtpcp, tlp, + lazy_timer); rtp = rtpcp->rtpp; raw_spin_lock_irqsave_rcu_node(rtpcp, flags); diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 1596812f7f12..b473ff056f49 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -985,7 +985,7 @@ static bool do_nocb_deferred_wakeup_common(struct rcu_data *rdp_gp, static void do_nocb_deferred_wakeup_timer(struct timer_list *t) { unsigned long flags; - struct rcu_data *rdp = from_timer(rdp, t, nocb_timer); + struct rcu_data *rdp = timer_container_of(rdp, t, nocb_timer); WARN_ON_ONCE(rdp->nocb_gp_rdp != rdp); trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("Timer")); diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index 56b21219442b..486c00536207 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -20,6 +20,28 @@ int sysctl_panic_on_rcu_stall __read_mostly; int sysctl_max_rcu_stall_to_panic __read_mostly; +#ifdef CONFIG_SYSFS + +static unsigned int rcu_stall_count; + +static ssize_t rcu_stall_count_show(struct kobject *kobj, struct kobj_attribute *attr, + char *page) +{ + return sysfs_emit(page, "%u\n", rcu_stall_count); +} + +static struct kobj_attribute rcu_stall_count_attr = __ATTR_RO(rcu_stall_count); + +static __init int kernel_rcu_stall_sysfs_init(void) +{ + sysfs_add_file_to_group(kernel_kobj, &rcu_stall_count_attr.attr, NULL); + return 0; +} + +late_initcall(kernel_rcu_stall_sysfs_init); + +#endif // CONFIG_SYSFS + #ifdef CONFIG_PROVE_RCU #define RCU_STALL_DELAY_DELTA (5 * HZ) #else @@ -784,6 +806,10 @@ static void check_cpu_stall(struct rcu_data *rdp) if (kvm_check_and_clear_guest_paused()) return; +#ifdef CONFIG_SYSFS + ++rcu_stall_count; +#endif + rcu_stall_notifier_call_chain(RCU_STALL_NOTIFY_NORM, (void *)j - gps); if (READ_ONCE(csd_lock_suppress_rcu_stall) && csd_lock_is_stuck()) { pr_err("INFO: %s detected stall, but suppressed full report due to a stuck CSD-lock.\n", rcu_state.name); diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 1396674fa722..ad04a5c3162a 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -733,7 +733,7 @@ static int psi_rtpoll_worker(void *data) static void poll_timer_fn(struct timer_list *t) { - struct psi_group *group = from_timer(group, t, rtpoll_timer); + struct psi_group *group = timer_container_of(group, t, rtpoll_timer); atomic_set(&group->rtpoll_wakeup, 1); wake_up_interruptible(&group->rtpoll_wait); diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c index 50e8d04ab661..2e5b89d7d866 100644 --- a/kernel/time/posix-cpu-timers.c +++ b/kernel/time/posix-cpu-timers.c @@ -1406,6 +1406,15 @@ void run_posix_cpu_timers(void) lockdep_assert_irqs_disabled(); /* + * Ensure that release_task(tsk) can't happen while + * handle_posix_cpu_timers() is running. Otherwise, a concurrent + * posix_cpu_timer_del() may fail to lock_task_sighand(tsk) and + * miss timer->it.cpu.firing != 0. + */ + if (tsk->exit_state) + return; + + /* * If the actual expiry is deferred to task work context and the * work is already scheduled there is no point to do anything here. */ diff --git a/kernel/time/sleep_timeout.c b/kernel/time/sleep_timeout.c index 5aa38b2cf40a..3c90574bd904 100644 --- a/kernel/time/sleep_timeout.c +++ b/kernel/time/sleep_timeout.c @@ -22,7 +22,7 @@ struct process_timer { static void process_timeout(struct timer_list *t) { - struct process_timer *timeout = from_timer(timeout, t, timer); + struct process_timer *timeout = timer_container_of(timeout, t, timer); wake_up_process(timeout->task); } diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index a7291685902e..4203fad56b6c 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -188,7 +188,7 @@ static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip, op->saved_func(ip, parent_ip, op, fregs); } -static void ftrace_sync_ipi(void *data) +void ftrace_sync_ipi(void *data) { /* Probably not needed, but do it anyway */ smp_rmb(); diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index e24509bd0af5..00fc38d70e86 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6795,7 +6795,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) old_size = buffer->subbuf_size; /* prevent another thread from changing buffer sizes */ - mutex_lock(&buffer->mutex); + guard(mutex)(&buffer->mutex); atomic_inc(&buffer->record_disabled); /* Make sure all commits have finished */ @@ -6900,7 +6900,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) } atomic_dec(&buffer->record_disabled); - mutex_unlock(&buffer->mutex); return 0; @@ -6909,7 +6908,6 @@ error: buffer->subbuf_size = old_size; atomic_dec(&buffer->record_disabled); - mutex_unlock(&buffer->mutex); for_each_buffer_cpu(buffer, cpu) { cpu_buffer = buffer->buffers[cpu]; diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c index 2048560264bb..08141f105c95 100644 --- a/kernel/trace/trace_events_filter.c +++ b/kernel/trace/trace_events_filter.c @@ -1250,7 +1250,9 @@ static void append_filter_err(struct trace_array *tr, static inline struct event_filter *event_filter(struct trace_event_file *file) { - return file->filter; + return rcu_dereference_protected(file->filter, + lockdep_is_held(&event_mutex)); + } /* caller must hold event_mutex */ @@ -1320,7 +1322,7 @@ void free_event_filter(struct event_filter *filter) static inline void __remove_filter(struct trace_event_file *file) { filter_disable(file); - remove_filter_string(file->filter); + remove_filter_string(event_filter(file)); } static void filter_free_subsystem_preds(struct trace_subsystem_dir *dir, @@ -1335,22 +1337,137 @@ static void filter_free_subsystem_preds(struct trace_subsystem_dir *dir, } } +struct filter_list { + struct list_head list; + struct event_filter *filter; +}; + +struct filter_head { + struct list_head list; + struct rcu_head rcu; +}; + + +static void free_filter_list(struct rcu_head *rhp) +{ + struct filter_head *filter_list = container_of(rhp, struct filter_head, rcu); + struct filter_list *filter_item, *tmp; + + list_for_each_entry_safe(filter_item, tmp, &filter_list->list, list) { + __free_filter(filter_item->filter); + list_del(&filter_item->list); + kfree(filter_item); + } + kfree(filter_list); +} + +static void free_filter_list_tasks(struct rcu_head *rhp) +{ + call_rcu(rhp, free_filter_list); +} + +/* + * The tracepoint_synchronize_unregister() is a double rcu call. + * It calls synchronize_rcu_tasks_trace() followed by synchronize_rcu(). + * Instead of waiting for it, simply call these via the call_rcu*() + * variants. + */ +static void delay_free_filter(struct filter_head *head) +{ + call_rcu_tasks_trace(&head->rcu, free_filter_list_tasks); +} + +static void try_delay_free_filter(struct event_filter *filter) +{ + struct filter_head *head; + struct filter_list *item; + + head = kmalloc(sizeof(*head), GFP_KERNEL); + if (!head) + goto free_now; + + INIT_LIST_HEAD(&head->list); + + item = kmalloc(sizeof(*item), GFP_KERNEL); + if (!item) { + kfree(head); + goto free_now; + } + + item->filter = filter; + list_add_tail(&item->list, &head->list); + delay_free_filter(head); + return; + + free_now: + /* Make sure the filter is not being used */ + tracepoint_synchronize_unregister(); + __free_filter(filter); +} + static inline void __free_subsystem_filter(struct trace_event_file *file) { - __free_filter(file->filter); + __free_filter(event_filter(file)); file->filter = NULL; } +static inline void event_set_filter(struct trace_event_file *file, + struct event_filter *filter) +{ + rcu_assign_pointer(file->filter, filter); +} + +static inline void event_clear_filter(struct trace_event_file *file) +{ + RCU_INIT_POINTER(file->filter, NULL); +} + static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir, - struct trace_array *tr) + struct trace_array *tr, + struct event_filter *filter) { struct trace_event_file *file; + struct filter_head *head; + struct filter_list *item; + + head = kmalloc(sizeof(*head), GFP_KERNEL); + if (!head) + goto free_now; + + INIT_LIST_HEAD(&head->list); + + item = kmalloc(sizeof(*item), GFP_KERNEL); + if (!item) + goto free_now; + + item->filter = filter; + list_add_tail(&item->list, &head->list); list_for_each_entry(file, &tr->events, list) { if (file->system != dir) continue; + item = kmalloc(sizeof(*item), GFP_KERNEL); + if (!item) + goto free_now; + item->filter = event_filter(file); + list_add_tail(&item->list, &head->list); + event_clear_filter(file); + } + + delay_free_filter(head); + return; + free_now: + tracepoint_synchronize_unregister(); + + if (head) + free_filter_list(&head->rcu); + + list_for_each_entry(file, &tr->events, list) { + if (file->system != dir || !file->filter) + continue; __free_subsystem_filter(file); } + __free_filter(filter); } int filter_assign_type(const char *type) @@ -2120,22 +2237,6 @@ static inline void event_set_filtered_flag(struct trace_event_file *file) trace_buffered_event_enable(); } -static inline void event_set_filter(struct trace_event_file *file, - struct event_filter *filter) -{ - rcu_assign_pointer(file->filter, filter); -} - -static inline void event_clear_filter(struct trace_event_file *file) -{ - RCU_INIT_POINTER(file->filter, NULL); -} - -struct filter_list { - struct list_head list; - struct event_filter *filter; -}; - static int process_system_preds(struct trace_subsystem_dir *dir, struct trace_array *tr, struct filter_parse_error *pe, @@ -2144,11 +2245,16 @@ static int process_system_preds(struct trace_subsystem_dir *dir, struct trace_event_file *file; struct filter_list *filter_item; struct event_filter *filter = NULL; - struct filter_list *tmp; - LIST_HEAD(filter_list); + struct filter_head *filter_list; bool fail = true; int err; + filter_list = kmalloc(sizeof(*filter_list), GFP_KERNEL); + if (!filter_list) + return -ENOMEM; + + INIT_LIST_HEAD(&filter_list->list); + list_for_each_entry(file, &tr->events, list) { if (file->system != dir) @@ -2175,7 +2281,7 @@ static int process_system_preds(struct trace_subsystem_dir *dir, if (!filter_item) goto fail_mem; - list_add_tail(&filter_item->list, &filter_list); + list_add_tail(&filter_item->list, &filter_list->list); /* * Regardless of if this returned an error, we still * replace the filter for the call. @@ -2195,31 +2301,22 @@ static int process_system_preds(struct trace_subsystem_dir *dir, * Do a synchronize_rcu() and to ensure all calls are * done with them before we free them. */ - tracepoint_synchronize_unregister(); - list_for_each_entry_safe(filter_item, tmp, &filter_list, list) { - __free_filter(filter_item->filter); - list_del(&filter_item->list); - kfree(filter_item); - } + delay_free_filter(filter_list); return 0; fail: /* No call succeeded */ - list_for_each_entry_safe(filter_item, tmp, &filter_list, list) { - list_del(&filter_item->list); - kfree(filter_item); - } + free_filter_list(&filter_list->rcu); parse_error(pe, FILT_ERR_BAD_SUBSYS_FILTER, 0); return -EINVAL; fail_mem: __free_filter(filter); + /* If any call succeeded, we still need to sync */ if (!fail) - tracepoint_synchronize_unregister(); - list_for_each_entry_safe(filter_item, tmp, &filter_list, list) { - __free_filter(filter_item->filter); - list_del(&filter_item->list); - kfree(filter_item); - } + delay_free_filter(filter_list); + else + free_filter_list(&filter_list->rcu); + return -ENOMEM; } @@ -2361,9 +2458,7 @@ int apply_event_filter(struct trace_event_file *file, char *filter_string) event_clear_filter(file); - /* Make sure the filter is not being used */ - tracepoint_synchronize_unregister(); - __free_filter(filter); + try_delay_free_filter(filter); return 0; } @@ -2387,11 +2482,8 @@ int apply_event_filter(struct trace_event_file *file, char *filter_string) event_set_filter(file, filter); - if (tmp) { - /* Make sure the call is done with the filter */ - tracepoint_synchronize_unregister(); - __free_filter(tmp); - } + if (tmp) + try_delay_free_filter(tmp); } return err; @@ -2417,9 +2509,7 @@ int apply_subsystem_event_filter(struct trace_subsystem_dir *dir, filter = system->filter; system->filter = NULL; /* Ensure all filters are no longer used */ - tracepoint_synchronize_unregister(); - filter_free_subsystem_filters(dir, tr); - __free_filter(filter); + filter_free_subsystem_filters(dir, tr, filter); return 0; } diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 3bef0754cf73..97f37b5bae66 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2481,7 +2481,7 @@ EXPORT_SYMBOL_GPL(queue_work_node); void delayed_work_timer_fn(struct timer_list *t) { - struct delayed_work *dwork = from_timer(dwork, t, timer); + struct delayed_work *dwork = timer_container_of(dwork, t, timer); /* should have been called from irqsafe timer with irq already off */ __queue_work(dwork->cpu, dwork->wq, &dwork->work); @@ -2909,7 +2909,7 @@ static void set_worker_dying(struct worker *worker, struct list_head *list) */ static void idle_worker_timeout(struct timer_list *t) { - struct worker_pool *pool = from_timer(pool, t, idle_timer); + struct worker_pool *pool = timer_container_of(pool, t, idle_timer); bool do_cull = false; if (work_pending(&pool->idle_cull_work)) @@ -3008,7 +3008,7 @@ static void send_mayday(struct work_struct *work) static void pool_mayday_timeout(struct timer_list *t) { - struct worker_pool *pool = from_timer(pool, t, mayday_timer); + struct worker_pool *pool = timer_container_of(pool, t, mayday_timer); struct work_struct *work; raw_spin_lock_irq(&pool->lock); diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 45dae7da70e1..d48b80f3f007 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -607,15 +607,16 @@ out: mas_unlock(&mas); } -static void load_module(struct module *mod, struct codetag *start, struct codetag *stop) +static int load_module(struct module *mod, struct codetag *start, struct codetag *stop) { /* Allocate module alloc_tag percpu counters */ struct alloc_tag *start_tag; struct alloc_tag *stop_tag; struct alloc_tag *tag; + /* percpu counters for core allocations are already statically allocated */ if (!mod) - return; + return 0; start_tag = ct_to_alloc_tag(start); stop_tag = ct_to_alloc_tag(stop); @@ -627,12 +628,13 @@ static void load_module(struct module *mod, struct codetag *start, struct codeta free_percpu(tag->counters); tag->counters = NULL; } - shutdown_mem_profiling(true); - pr_err("Failed to allocate memory for allocation tag percpu counters in the module %s. Memory allocation profiling is disabled!\n", + pr_err("Failed to allocate memory for allocation tag percpu counters in the module %s\n", mod->name); - break; + return -ENOMEM; } } + + return 0; } static void replace_module(struct module *mod, struct module *new_mod) diff --git a/lib/codetag.c b/lib/codetag.c index de332e98d6f5..650d54d7e14d 100644 --- a/lib/codetag.c +++ b/lib/codetag.c @@ -167,6 +167,7 @@ static int codetag_module_init(struct codetag_type *cttype, struct module *mod) { struct codetag_range range; struct codetag_module *cmod; + int mod_id; int err; range = get_section_range(mod, cttype->desc.section); @@ -190,11 +191,20 @@ static int codetag_module_init(struct codetag_type *cttype, struct module *mod) cmod->range = range; down_write(&cttype->mod_lock); - err = idr_alloc(&cttype->mod_idr, cmod, 0, 0, GFP_KERNEL); - if (err >= 0) { - cttype->count += range_size(cttype, &range); - if (cttype->desc.module_load) - cttype->desc.module_load(mod, range.start, range.stop); + mod_id = idr_alloc(&cttype->mod_idr, cmod, 0, 0, GFP_KERNEL); + if (mod_id >= 0) { + if (cttype->desc.module_load) { + err = cttype->desc.module_load(mod, range.start, range.stop); + if (!err) + cttype->count += range_size(cttype, &range); + else + idr_remove(&cttype->mod_idr, mod_id); + } else { + cttype->count += range_size(cttype, &range); + err = 0; + } + } else { + err = mod_id; } up_write(&cttype->mod_lock); @@ -295,17 +305,23 @@ void codetag_module_replaced(struct module *mod, struct module *new_mod) mutex_unlock(&codetag_lock); } -void codetag_load_module(struct module *mod) +int codetag_load_module(struct module *mod) { struct codetag_type *cttype; + int ret = 0; if (!mod) - return; + return 0; mutex_lock(&codetag_lock); - list_for_each_entry(cttype, &codetag_types, link) - codetag_module_init(cttype, mod); + list_for_each_entry(cttype, &codetag_types, link) { + ret = codetag_module_init(cttype, mod); + if (ret) + break; + } mutex_unlock(&codetag_lock); + + return ret; } void codetag_unload_module(struct module *mod) diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 969d4ad510df..f9193f952f49 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -817,7 +817,7 @@ static bool iov_iter_aligned_bvec(const struct iov_iter *i, unsigned addr_mask, size_t size = i->count; do { - size_t len = bvec->bv_len; + size_t len = bvec->bv_len - skip; if (len > size) len = size; diff --git a/lib/raid6/Makefile b/lib/raid6/Makefile index 29127dd05d63..5be0a4e60ab1 100644 --- a/lib/raid6/Makefile +++ b/lib/raid6/Makefile @@ -10,6 +10,7 @@ raid6_pq-$(CONFIG_ALTIVEC) += altivec1.o altivec2.o altivec4.o altivec8.o \ raid6_pq-$(CONFIG_KERNEL_MODE_NEON) += neon.o neon1.o neon2.o neon4.o neon8.o recov_neon.o recov_neon_inner.o raid6_pq-$(CONFIG_S390) += s390vx8.o recov_s390xc.o raid6_pq-$(CONFIG_LOONGARCH) += loongarch_simd.o recov_loongarch_simd.o +raid6_pq-$(CONFIG_RISCV_ISA_V) += rvv.o recov_rvv.o hostprogs += mktables diff --git a/lib/raid6/algos.c b/lib/raid6/algos.c index dfd3f800ac9b..75ce3e134b7c 100644 --- a/lib/raid6/algos.c +++ b/lib/raid6/algos.c @@ -77,6 +77,12 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_lsx, #endif #endif +#ifdef CONFIG_RISCV_ISA_V + &raid6_rvvx1, + &raid6_rvvx2, + &raid6_rvvx4, + &raid6_rvvx8, +#endif &raid6_intx8, &raid6_intx4, &raid6_intx2, @@ -110,6 +116,9 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = { &raid6_recov_lsx, #endif #endif +#ifdef CONFIG_RISCV_ISA_V + &raid6_recov_rvv, +#endif &raid6_recov_intx1, NULL }; diff --git a/lib/raid6/recov_rvv.c b/lib/raid6/recov_rvv.c new file mode 100644 index 000000000000..f29303795ccf --- /dev/null +++ b/lib/raid6/recov_rvv.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2024 Institute of Software, CAS. + * Author: Chunyan Zhang <zhangchunyan@iscas.ac.cn> + */ + +#include <asm/simd.h> +#include <asm/vector.h> +#include <crypto/internal/simd.h> +#include <linux/raid/pq.h> + +static int rvv_has_vector(void) +{ + return has_vector(); +} + +static void __raid6_2data_recov_rvv(int bytes, u8 *p, u8 *q, u8 *dp, + u8 *dq, const u8 *pbmul, + const u8 *qmul) +{ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli x0, %[avl], e8, m1, ta, ma\n" + ".option pop\n" + : : + [avl]"r"(16) + ); + + /* + * while ( bytes-- ) { + * uint8_t px, qx, db; + * + * px = *p ^ *dp; + * qx = qmul[*q ^ *dq]; + * *dq++ = db = pbmul[px] ^ qx; + * *dp++ = db ^ px; + * p++; q++; + * } + */ + while (bytes) { + /* + * v0:px, v1:dp, + * v2:qx, v3:dq, + * v4:vx, v5:vy, + * v6:qm0, v7:qm1, + * v8:pm0, v9:pm1, + * v14:p/qm[vx], v15:p/qm[vy] + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[px])\n" + "vle8.v v1, (%[dp])\n" + "vxor.vv v0, v0, v1\n" + "vle8.v v2, (%[qx])\n" + "vle8.v v3, (%[dq])\n" + "vxor.vv v4, v2, v3\n" + "vsrl.vi v5, v4, 4\n" + "vand.vi v4, v4, 0xf\n" + "vle8.v v6, (%[qm0])\n" + "vle8.v v7, (%[qm1])\n" + "vrgather.vv v14, v6, v4\n" /* v14 = qm[vx] */ + "vrgather.vv v15, v7, v5\n" /* v15 = qm[vy] */ + "vxor.vv v2, v14, v15\n" /* v2 = qmul[*q ^ *dq] */ + + "vsrl.vi v5, v0, 4\n" + "vand.vi v4, v0, 0xf\n" + "vle8.v v8, (%[pm0])\n" + "vle8.v v9, (%[pm1])\n" + "vrgather.vv v14, v8, v4\n" /* v14 = pm[vx] */ + "vrgather.vv v15, v9, v5\n" /* v15 = pm[vy] */ + "vxor.vv v4, v14, v15\n" /* v4 = pbmul[px] */ + "vxor.vv v3, v4, v2\n" /* v3 = db = pbmul[px] ^ qx */ + "vxor.vv v1, v3, v0\n" /* v1 = db ^ px; */ + "vse8.v v3, (%[dq])\n" + "vse8.v v1, (%[dp])\n" + ".option pop\n" + : : + [px]"r"(p), + [dp]"r"(dp), + [qx]"r"(q), + [dq]"r"(dq), + [qm0]"r"(qmul), + [qm1]"r"(qmul + 16), + [pm0]"r"(pbmul), + [pm1]"r"(pbmul + 16) + :); + + bytes -= 16; + p += 16; + q += 16; + dp += 16; + dq += 16; + } +} + +static void __raid6_datap_recov_rvv(int bytes, u8 *p, u8 *q, + u8 *dq, const u8 *qmul) +{ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli x0, %[avl], e8, m1, ta, ma\n" + ".option pop\n" + : : + [avl]"r"(16) + ); + + /* + * while (bytes--) { + * *p++ ^= *dq = qmul[*q ^ *dq]; + * q++; dq++; + * } + */ + while (bytes) { + /* + * v0:vx, v1:vy, + * v2:dq, v3:p, + * v4:qm0, v5:qm1, + * v10:m[vx], v11:m[vy] + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[vx])\n" + "vle8.v v2, (%[dq])\n" + "vxor.vv v0, v0, v2\n" + "vsrl.vi v1, v0, 4\n" + "vand.vi v0, v0, 0xf\n" + "vle8.v v4, (%[qm0])\n" + "vle8.v v5, (%[qm1])\n" + "vrgather.vv v10, v4, v0\n" + "vrgather.vv v11, v5, v1\n" + "vxor.vv v0, v10, v11\n" + "vle8.v v1, (%[vy])\n" + "vxor.vv v1, v0, v1\n" + "vse8.v v0, (%[dq])\n" + "vse8.v v1, (%[vy])\n" + ".option pop\n" + : : + [vx]"r"(q), + [vy]"r"(p), + [dq]"r"(dq), + [qm0]"r"(qmul), + [qm1]"r"(qmul + 16) + :); + + bytes -= 16; + p += 16; + q += 16; + dq += 16; + } +} + +static void raid6_2data_recov_rvv(int disks, size_t bytes, int faila, + int failb, void **ptrs) +{ + u8 *p, *q, *dp, *dq; + const u8 *pbmul; /* P multiplier table for B data */ + const u8 *qmul; /* Q multiplier table (for both) */ + + p = (u8 *)ptrs[disks - 2]; + q = (u8 *)ptrs[disks - 1]; + + /* + * Compute syndrome with zero for the missing data pages + * Use the dead data pages as temporary storage for + * delta p and delta q + */ + dp = (u8 *)ptrs[faila]; + ptrs[faila] = (void *)raid6_empty_zero_page; + ptrs[disks - 2] = dp; + dq = (u8 *)ptrs[failb]; + ptrs[failb] = (void *)raid6_empty_zero_page; + ptrs[disks - 1] = dq; + + raid6_call.gen_syndrome(disks, bytes, ptrs); + + /* Restore pointer table */ + ptrs[faila] = dp; + ptrs[failb] = dq; + ptrs[disks - 2] = p; + ptrs[disks - 1] = q; + + /* Now, pick the proper data tables */ + pbmul = raid6_vgfmul[raid6_gfexi[failb - faila]]; + qmul = raid6_vgfmul[raid6_gfinv[raid6_gfexp[faila] ^ + raid6_gfexp[failb]]]; + + kernel_vector_begin(); + __raid6_2data_recov_rvv(bytes, p, q, dp, dq, pbmul, qmul); + kernel_vector_end(); +} + +static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila, + void **ptrs) +{ + u8 *p, *q, *dq; + const u8 *qmul; /* Q multiplier table */ + + p = (u8 *)ptrs[disks - 2]; + q = (u8 *)ptrs[disks - 1]; + + /* + * Compute syndrome with zero for the missing data page + * Use the dead data page as temporary storage for delta q + */ + dq = (u8 *)ptrs[faila]; + ptrs[faila] = (void *)raid6_empty_zero_page; + ptrs[disks - 1] = dq; + + raid6_call.gen_syndrome(disks, bytes, ptrs); + + /* Restore pointer table */ + ptrs[faila] = dq; + ptrs[disks - 1] = q; + + /* Now, pick the proper data tables */ + qmul = raid6_vgfmul[raid6_gfinv[raid6_gfexp[faila]]]; + + kernel_vector_begin(); + __raid6_datap_recov_rvv(bytes, p, q, dq, qmul); + kernel_vector_end(); +} + +const struct raid6_recov_calls raid6_recov_rvv = { + .data2 = raid6_2data_recov_rvv, + .datap = raid6_datap_recov_rvv, + .valid = rvv_has_vector, + .name = "rvv", + .priority = 1, +}; diff --git a/lib/raid6/rvv.c b/lib/raid6/rvv.c new file mode 100644 index 000000000000..f0887344b274 --- /dev/null +++ b/lib/raid6/rvv.c @@ -0,0 +1,1212 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * RAID-6 syndrome calculation using RISC-V vector instructions + * + * Copyright 2024 Institute of Software, CAS. + * Author: Chunyan Zhang <zhangchunyan@iscas.ac.cn> + * + * Based on neon.uc: + * Copyright 2002-2004 H. Peter Anvin + */ + +#include <asm/simd.h> +#include <asm/vector.h> +#include <crypto/internal/simd.h> +#include <linux/raid/pq.h> +#include <linux/types.h> +#include "rvv.h" + +#define NSIZE (riscv_v_vsize / 32) /* NSIZE = vlenb */ + +static int rvv_has_vector(void) +{ + return has_vector(); +} + +static void raid6_rvv1_gen_syndrome_real(int disks, unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + unsigned long d; + int z, z0; + u8 *p, *q; + + z0 = disks - 3; /* Highest data disk */ + p = dptr[z0 + 1]; /* XOR parity */ + q = dptr[z0 + 2]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 */ + for (d = 0; d < bytes; d += NSIZE * 1) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]) + ); + + for (z = z0 - 1 ; z >= 0 ; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] = wp$$; + * *(unative_t *)&q[d+NSIZE*$$] = wq$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vse8.v v0, (%[wp0])\n" + "vse8.v v1, (%[wq0])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]) + ); + } +} + +static void raid6_rvv1_xor_syndrome_real(int disks, int start, int stop, + unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + u8 *p, *q; + unsigned long d; + int z, z0; + + z0 = stop; /* P/Q right side optimization */ + p = dptr[disks - 2]; /* XOR parity */ + q = dptr[disks - 1]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 */ + for (d = 0 ; d < bytes ; d += NSIZE * 1) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]) + ); + + /* P/Q data pages */ + for (z = z0 - 1; z >= start; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* P/Q left side optimization */ + for (z = start - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * wq$$ = w1$$ ^ w2$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v1, v3, v2\n" + ".option pop\n" + : : + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] ^= wp$$; + * *(unative_t *)&q[d+NSIZE*$$] ^= wq$$; + * v0:wp0, v1:wq0, v2:p0, v3:q0 + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v2, (%[wp0])\n" + "vle8.v v3, (%[wq0])\n" + "vxor.vv v2, v2, v0\n" + "vxor.vv v3, v3, v1\n" + "vse8.v v2, (%[wp0])\n" + "vse8.v v3, (%[wq0])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]) + ); + } +} + +static void raid6_rvv2_gen_syndrome_real(int disks, unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + unsigned long d; + int z, z0; + u8 *p, *q; + + z0 = disks - 3; /* Highest data disk */ + p = dptr[z0 + 1]; /* XOR parity */ + q = dptr[z0 + 2]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + */ + for (d = 0; d < bytes; d += NSIZE * 2) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]) + ); + + for (z = z0 - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] = wp$$; + * *(unative_t *)&q[d+NSIZE*$$] = wq$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vse8.v v0, (%[wp0])\n" + "vse8.v v1, (%[wq0])\n" + "vse8.v v4, (%[wp1])\n" + "vse8.v v5, (%[wq1])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]) + ); + } +} + +static void raid6_rvv2_xor_syndrome_real(int disks, int start, int stop, + unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + u8 *p, *q; + unsigned long d; + int z, z0; + + z0 = stop; /* P/Q right side optimization */ + p = dptr[disks - 2]; /* XOR parity */ + q = dptr[disks - 1]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + */ + for (d = 0; d < bytes; d += NSIZE * 2) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]) + ); + + /* P/Q data pages */ + for (z = z0 - 1; z >= start; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* P/Q left side optimization */ + for (z = start - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * wq$$ = w1$$ ^ w2$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v1, v3, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v5, v7, v6\n" + ".option pop\n" + : : + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] ^= wp$$; + * *(unative_t *)&q[d+NSIZE*$$] ^= wq$$; + * v0:wp0, v1:wq0, v2:p0, v3:q0 + * v4:wp1, v5:wq1, v6:p1, v7:q1 + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v2, (%[wp0])\n" + "vle8.v v3, (%[wq0])\n" + "vxor.vv v2, v2, v0\n" + "vxor.vv v3, v3, v1\n" + "vse8.v v2, (%[wp0])\n" + "vse8.v v3, (%[wq0])\n" + + "vle8.v v6, (%[wp1])\n" + "vle8.v v7, (%[wq1])\n" + "vxor.vv v6, v6, v4\n" + "vxor.vv v7, v7, v5\n" + "vse8.v v6, (%[wp1])\n" + "vse8.v v7, (%[wq1])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]) + ); + } +} + +static void raid6_rvv4_gen_syndrome_real(int disks, unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + unsigned long d; + int z, z0; + u8 *p, *q; + + z0 = disks - 3; /* Highest data disk */ + p = dptr[z0 + 1]; /* XOR parity */ + q = dptr[z0 + 2]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + * v8:wp2, v9:wq2, v10:wd2/w22, v11:w12 + * v12:wp3, v13:wq3, v14:wd3/w23, v15:w13 + */ + for (d = 0; d < bytes; d += NSIZE * 4) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + "vle8.v v8, (%[wp2])\n" + "vle8.v v9, (%[wp2])\n" + "vle8.v v12, (%[wp3])\n" + "vle8.v v13, (%[wp3])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]), + [wp2]"r"(&dptr[z0][d + 2 * NSIZE]), + [wp3]"r"(&dptr[z0][d + 3 * NSIZE]) + ); + + for (z = z0 - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v11, v11, v10\n" + "vle8.v v10, (%[wd2])\n" + "vxor.vv v9, v11, v10\n" + "vxor.vv v8, v8, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v15, v15, v14\n" + "vle8.v v14, (%[wd3])\n" + "vxor.vv v13, v15, v14\n" + "vxor.vv v12, v12, v14\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [wd2]"r"(&dptr[z][d + 2 * NSIZE]), + [wd3]"r"(&dptr[z][d + 3 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] = wp$$; + * *(unative_t *)&q[d+NSIZE*$$] = wq$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vse8.v v0, (%[wp0])\n" + "vse8.v v1, (%[wq0])\n" + "vse8.v v4, (%[wp1])\n" + "vse8.v v5, (%[wq1])\n" + "vse8.v v8, (%[wp2])\n" + "vse8.v v9, (%[wq2])\n" + "vse8.v v12, (%[wp3])\n" + "vse8.v v13, (%[wq3])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]), + [wp2]"r"(&p[d + NSIZE * 2]), + [wq2]"r"(&q[d + NSIZE * 2]), + [wp3]"r"(&p[d + NSIZE * 3]), + [wq3]"r"(&q[d + NSIZE * 3]) + ); + } +} + +static void raid6_rvv4_xor_syndrome_real(int disks, int start, int stop, + unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + u8 *p, *q; + unsigned long d; + int z, z0; + + z0 = stop; /* P/Q right side optimization */ + p = dptr[disks - 2]; /* XOR parity */ + q = dptr[disks - 1]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + * v8:wp2, v9:wq2, v10:wd2/w22, v11:w12 + * v12:wp3, v13:wq3, v14:wd3/w23, v15:w13 + */ + for (d = 0; d < bytes; d += NSIZE * 4) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + "vle8.v v8, (%[wp2])\n" + "vle8.v v9, (%[wp2])\n" + "vle8.v v12, (%[wp3])\n" + "vle8.v v13, (%[wp3])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]), + [wp2]"r"(&dptr[z0][d + 2 * NSIZE]), + [wp3]"r"(&dptr[z0][d + 3 * NSIZE]) + ); + + /* P/Q data pages */ + for (z = z0 - 1; z >= start; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v11, v11, v10\n" + "vle8.v v10, (%[wd2])\n" + "vxor.vv v9, v11, v10\n" + "vxor.vv v8, v8, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v15, v15, v14\n" + "vle8.v v14, (%[wd3])\n" + "vxor.vv v13, v15, v14\n" + "vxor.vv v12, v12, v14\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [wd2]"r"(&dptr[z][d + 2 * NSIZE]), + [wd3]"r"(&dptr[z][d + 3 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* P/Q left side optimization */ + for (z = start - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * wq$$ = w1$$ ^ w2$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v1, v3, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v5, v7, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v9, v11, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v13, v15, v14\n" + ".option pop\n" + : : + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] ^= wp$$; + * *(unative_t *)&q[d+NSIZE*$$] ^= wq$$; + * v0:wp0, v1:wq0, v2:p0, v3:q0 + * v4:wp1, v5:wq1, v6:p1, v7:q1 + * v8:wp2, v9:wq2, v10:p2, v11:q2 + * v12:wp3, v13:wq3, v14:p3, v15:q3 + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v2, (%[wp0])\n" + "vle8.v v3, (%[wq0])\n" + "vxor.vv v2, v2, v0\n" + "vxor.vv v3, v3, v1\n" + "vse8.v v2, (%[wp0])\n" + "vse8.v v3, (%[wq0])\n" + + "vle8.v v6, (%[wp1])\n" + "vle8.v v7, (%[wq1])\n" + "vxor.vv v6, v6, v4\n" + "vxor.vv v7, v7, v5\n" + "vse8.v v6, (%[wp1])\n" + "vse8.v v7, (%[wq1])\n" + + "vle8.v v10, (%[wp2])\n" + "vle8.v v11, (%[wq2])\n" + "vxor.vv v10, v10, v8\n" + "vxor.vv v11, v11, v9\n" + "vse8.v v10, (%[wp2])\n" + "vse8.v v11, (%[wq2])\n" + + "vle8.v v14, (%[wp3])\n" + "vle8.v v15, (%[wq3])\n" + "vxor.vv v14, v14, v12\n" + "vxor.vv v15, v15, v13\n" + "vse8.v v14, (%[wp3])\n" + "vse8.v v15, (%[wq3])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]), + [wp2]"r"(&p[d + NSIZE * 2]), + [wq2]"r"(&q[d + NSIZE * 2]), + [wp3]"r"(&p[d + NSIZE * 3]), + [wq3]"r"(&q[d + NSIZE * 3]) + ); + } +} + +static void raid6_rvv8_gen_syndrome_real(int disks, unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + unsigned long d; + int z, z0; + u8 *p, *q; + + z0 = disks - 3; /* Highest data disk */ + p = dptr[z0 + 1]; /* XOR parity */ + q = dptr[z0 + 2]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + * v8:wp2, v9:wq2, v10:wd2/w22, v11:w12 + * v12:wp3, v13:wq3, v14:wd3/w23, v15:w13 + * v16:wp4, v17:wq4, v18:wd4/w24, v19:w14 + * v20:wp5, v21:wq5, v22:wd5/w25, v23:w15 + * v24:wp6, v25:wq6, v26:wd6/w26, v27:w16 + * v28:wp7, v29:wq7, v30:wd7/w27, v31:w17 + */ + for (d = 0; d < bytes; d += NSIZE * 8) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + "vle8.v v8, (%[wp2])\n" + "vle8.v v9, (%[wp2])\n" + "vle8.v v12, (%[wp3])\n" + "vle8.v v13, (%[wp3])\n" + "vle8.v v16, (%[wp4])\n" + "vle8.v v17, (%[wp4])\n" + "vle8.v v20, (%[wp5])\n" + "vle8.v v21, (%[wp5])\n" + "vle8.v v24, (%[wp6])\n" + "vle8.v v25, (%[wp6])\n" + "vle8.v v28, (%[wp7])\n" + "vle8.v v29, (%[wp7])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]), + [wp2]"r"(&dptr[z0][d + 2 * NSIZE]), + [wp3]"r"(&dptr[z0][d + 3 * NSIZE]), + [wp4]"r"(&dptr[z0][d + 4 * NSIZE]), + [wp5]"r"(&dptr[z0][d + 5 * NSIZE]), + [wp6]"r"(&dptr[z0][d + 6 * NSIZE]), + [wp7]"r"(&dptr[z0][d + 7 * NSIZE]) + ); + + for (z = z0 - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v11, v11, v10\n" + "vle8.v v10, (%[wd2])\n" + "vxor.vv v9, v11, v10\n" + "vxor.vv v8, v8, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v15, v15, v14\n" + "vle8.v v14, (%[wd3])\n" + "vxor.vv v13, v15, v14\n" + "vxor.vv v12, v12, v14\n" + + "vsra.vi v18, v17, 7\n" + "vsll.vi v19, v17, 1\n" + "vand.vx v18, v18, %[x1d]\n" + "vxor.vv v19, v19, v18\n" + "vle8.v v18, (%[wd4])\n" + "vxor.vv v17, v19, v18\n" + "vxor.vv v16, v16, v18\n" + + "vsra.vi v22, v21, 7\n" + "vsll.vi v23, v21, 1\n" + "vand.vx v22, v22, %[x1d]\n" + "vxor.vv v23, v23, v22\n" + "vle8.v v22, (%[wd5])\n" + "vxor.vv v21, v23, v22\n" + "vxor.vv v20, v20, v22\n" + + "vsra.vi v26, v25, 7\n" + "vsll.vi v27, v25, 1\n" + "vand.vx v26, v26, %[x1d]\n" + "vxor.vv v27, v27, v26\n" + "vle8.v v26, (%[wd6])\n" + "vxor.vv v25, v27, v26\n" + "vxor.vv v24, v24, v26\n" + + "vsra.vi v30, v29, 7\n" + "vsll.vi v31, v29, 1\n" + "vand.vx v30, v30, %[x1d]\n" + "vxor.vv v31, v31, v30\n" + "vle8.v v30, (%[wd7])\n" + "vxor.vv v29, v31, v30\n" + "vxor.vv v28, v28, v30\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [wd2]"r"(&dptr[z][d + 2 * NSIZE]), + [wd3]"r"(&dptr[z][d + 3 * NSIZE]), + [wd4]"r"(&dptr[z][d + 4 * NSIZE]), + [wd5]"r"(&dptr[z][d + 5 * NSIZE]), + [wd6]"r"(&dptr[z][d + 6 * NSIZE]), + [wd7]"r"(&dptr[z][d + 7 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] = wp$$; + * *(unative_t *)&q[d+NSIZE*$$] = wq$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vse8.v v0, (%[wp0])\n" + "vse8.v v1, (%[wq0])\n" + "vse8.v v4, (%[wp1])\n" + "vse8.v v5, (%[wq1])\n" + "vse8.v v8, (%[wp2])\n" + "vse8.v v9, (%[wq2])\n" + "vse8.v v12, (%[wp3])\n" + "vse8.v v13, (%[wq3])\n" + "vse8.v v16, (%[wp4])\n" + "vse8.v v17, (%[wq4])\n" + "vse8.v v20, (%[wp5])\n" + "vse8.v v21, (%[wq5])\n" + "vse8.v v24, (%[wp6])\n" + "vse8.v v25, (%[wq6])\n" + "vse8.v v28, (%[wp7])\n" + "vse8.v v29, (%[wq7])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]), + [wp2]"r"(&p[d + NSIZE * 2]), + [wq2]"r"(&q[d + NSIZE * 2]), + [wp3]"r"(&p[d + NSIZE * 3]), + [wq3]"r"(&q[d + NSIZE * 3]), + [wp4]"r"(&p[d + NSIZE * 4]), + [wq4]"r"(&q[d + NSIZE * 4]), + [wp5]"r"(&p[d + NSIZE * 5]), + [wq5]"r"(&q[d + NSIZE * 5]), + [wp6]"r"(&p[d + NSIZE * 6]), + [wq6]"r"(&q[d + NSIZE * 6]), + [wp7]"r"(&p[d + NSIZE * 7]), + [wq7]"r"(&q[d + NSIZE * 7]) + ); + } +} + +static void raid6_rvv8_xor_syndrome_real(int disks, int start, int stop, + unsigned long bytes, void **ptrs) +{ + u8 **dptr = (u8 **)ptrs; + u8 *p, *q; + unsigned long d; + int z, z0; + + z0 = stop; /* P/Q right side optimization */ + p = dptr[disks - 2]; /* XOR parity */ + q = dptr[disks - 1]; /* RS syndrome */ + + asm volatile (".option push\n" + ".option arch,+v\n" + "vsetvli t0, x0, e8, m1, ta, ma\n" + ".option pop\n" + ); + + /* + * v0:wp0, v1:wq0, v2:wd0/w20, v3:w10 + * v4:wp1, v5:wq1, v6:wd1/w21, v7:w11 + * v8:wp2, v9:wq2, v10:wd2/w22, v11:w12 + * v12:wp3, v13:wq3, v14:wd3/w23, v15:w13 + * v16:wp4, v17:wq4, v18:wd4/w24, v19:w14 + * v20:wp5, v21:wq5, v22:wd5/w25, v23:w15 + * v24:wp6, v25:wq6, v26:wd6/w26, v27:w16 + * v28:wp7, v29:wq7, v30:wd7/w27, v31:w17 + */ + for (d = 0; d < bytes; d += NSIZE * 8) { + /* wq$$ = wp$$ = *(unative_t *)&dptr[z0][d+$$*NSIZE]; */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v0, (%[wp0])\n" + "vle8.v v1, (%[wp0])\n" + "vle8.v v4, (%[wp1])\n" + "vle8.v v5, (%[wp1])\n" + "vle8.v v8, (%[wp2])\n" + "vle8.v v9, (%[wp2])\n" + "vle8.v v12, (%[wp3])\n" + "vle8.v v13, (%[wp3])\n" + "vle8.v v16, (%[wp4])\n" + "vle8.v v17, (%[wp4])\n" + "vle8.v v20, (%[wp5])\n" + "vle8.v v21, (%[wp5])\n" + "vle8.v v24, (%[wp6])\n" + "vle8.v v25, (%[wp6])\n" + "vle8.v v28, (%[wp7])\n" + "vle8.v v29, (%[wp7])\n" + ".option pop\n" + : : + [wp0]"r"(&dptr[z0][d + 0 * NSIZE]), + [wp1]"r"(&dptr[z0][d + 1 * NSIZE]), + [wp2]"r"(&dptr[z0][d + 2 * NSIZE]), + [wp3]"r"(&dptr[z0][d + 3 * NSIZE]), + [wp4]"r"(&dptr[z0][d + 4 * NSIZE]), + [wp5]"r"(&dptr[z0][d + 5 * NSIZE]), + [wp6]"r"(&dptr[z0][d + 6 * NSIZE]), + [wp7]"r"(&dptr[z0][d + 7 * NSIZE]) + ); + + /* P/Q data pages */ + for (z = z0 - 1; z >= start; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * w1$$ ^= w2$$; + * wd$$ = *(unative_t *)&dptr[z][d+$$*NSIZE]; + * wq$$ = w1$$ ^ wd$$; + * wp$$ ^= wd$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v3, v3, v2\n" + "vle8.v v2, (%[wd0])\n" + "vxor.vv v1, v3, v2\n" + "vxor.vv v0, v0, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v7, v7, v6\n" + "vle8.v v6, (%[wd1])\n" + "vxor.vv v5, v7, v6\n" + "vxor.vv v4, v4, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v11, v11, v10\n" + "vle8.v v10, (%[wd2])\n" + "vxor.vv v9, v11, v10\n" + "vxor.vv v8, v8, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v15, v15, v14\n" + "vle8.v v14, (%[wd3])\n" + "vxor.vv v13, v15, v14\n" + "vxor.vv v12, v12, v14\n" + + "vsra.vi v18, v17, 7\n" + "vsll.vi v19, v17, 1\n" + "vand.vx v18, v18, %[x1d]\n" + "vxor.vv v19, v19, v18\n" + "vle8.v v18, (%[wd4])\n" + "vxor.vv v17, v19, v18\n" + "vxor.vv v16, v16, v18\n" + + "vsra.vi v22, v21, 7\n" + "vsll.vi v23, v21, 1\n" + "vand.vx v22, v22, %[x1d]\n" + "vxor.vv v23, v23, v22\n" + "vle8.v v22, (%[wd5])\n" + "vxor.vv v21, v23, v22\n" + "vxor.vv v20, v20, v22\n" + + "vsra.vi v26, v25, 7\n" + "vsll.vi v27, v25, 1\n" + "vand.vx v26, v26, %[x1d]\n" + "vxor.vv v27, v27, v26\n" + "vle8.v v26, (%[wd6])\n" + "vxor.vv v25, v27, v26\n" + "vxor.vv v24, v24, v26\n" + + "vsra.vi v30, v29, 7\n" + "vsll.vi v31, v29, 1\n" + "vand.vx v30, v30, %[x1d]\n" + "vxor.vv v31, v31, v30\n" + "vle8.v v30, (%[wd7])\n" + "vxor.vv v29, v31, v30\n" + "vxor.vv v28, v28, v30\n" + ".option pop\n" + : : + [wd0]"r"(&dptr[z][d + 0 * NSIZE]), + [wd1]"r"(&dptr[z][d + 1 * NSIZE]), + [wd2]"r"(&dptr[z][d + 2 * NSIZE]), + [wd3]"r"(&dptr[z][d + 3 * NSIZE]), + [wd4]"r"(&dptr[z][d + 4 * NSIZE]), + [wd5]"r"(&dptr[z][d + 5 * NSIZE]), + [wd6]"r"(&dptr[z][d + 6 * NSIZE]), + [wd7]"r"(&dptr[z][d + 7 * NSIZE]), + [x1d]"r"(0x1d) + ); + } + + /* P/Q left side optimization */ + for (z = start - 1; z >= 0; z--) { + /* + * w2$$ = MASK(wq$$); + * w1$$ = SHLBYTE(wq$$); + * w2$$ &= NBYTES(0x1d); + * wq$$ = w1$$ ^ w2$$; + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vsra.vi v2, v1, 7\n" + "vsll.vi v3, v1, 1\n" + "vand.vx v2, v2, %[x1d]\n" + "vxor.vv v1, v3, v2\n" + + "vsra.vi v6, v5, 7\n" + "vsll.vi v7, v5, 1\n" + "vand.vx v6, v6, %[x1d]\n" + "vxor.vv v5, v7, v6\n" + + "vsra.vi v10, v9, 7\n" + "vsll.vi v11, v9, 1\n" + "vand.vx v10, v10, %[x1d]\n" + "vxor.vv v9, v11, v10\n" + + "vsra.vi v14, v13, 7\n" + "vsll.vi v15, v13, 1\n" + "vand.vx v14, v14, %[x1d]\n" + "vxor.vv v13, v15, v14\n" + + "vsra.vi v18, v17, 7\n" + "vsll.vi v19, v17, 1\n" + "vand.vx v18, v18, %[x1d]\n" + "vxor.vv v17, v19, v18\n" + + "vsra.vi v22, v21, 7\n" + "vsll.vi v23, v21, 1\n" + "vand.vx v22, v22, %[x1d]\n" + "vxor.vv v21, v23, v22\n" + + "vsra.vi v26, v25, 7\n" + "vsll.vi v27, v25, 1\n" + "vand.vx v26, v26, %[x1d]\n" + "vxor.vv v25, v27, v26\n" + + "vsra.vi v30, v29, 7\n" + "vsll.vi v31, v29, 1\n" + "vand.vx v30, v30, %[x1d]\n" + "vxor.vv v29, v31, v30\n" + ".option pop\n" + : : + [x1d]"r"(0x1d) + ); + } + + /* + * *(unative_t *)&p[d+NSIZE*$$] ^= wp$$; + * *(unative_t *)&q[d+NSIZE*$$] ^= wq$$; + * v0:wp0, v1:wq0, v2:p0, v3:q0 + * v4:wp1, v5:wq1, v6:p1, v7:q1 + * v8:wp2, v9:wq2, v10:p2, v11:q2 + * v12:wp3, v13:wq3, v14:p3, v15:q3 + * v16:wp4, v17:wq4, v18:p4, v19:q4 + * v20:wp5, v21:wq5, v22:p5, v23:q5 + * v24:wp6, v25:wq6, v26:p6, v27:q6 + * v28:wp7, v29:wq7, v30:p7, v31:q7 + */ + asm volatile (".option push\n" + ".option arch,+v\n" + "vle8.v v2, (%[wp0])\n" + "vle8.v v3, (%[wq0])\n" + "vxor.vv v2, v2, v0\n" + "vxor.vv v3, v3, v1\n" + "vse8.v v2, (%[wp0])\n" + "vse8.v v3, (%[wq0])\n" + + "vle8.v v6, (%[wp1])\n" + "vle8.v v7, (%[wq1])\n" + "vxor.vv v6, v6, v4\n" + "vxor.vv v7, v7, v5\n" + "vse8.v v6, (%[wp1])\n" + "vse8.v v7, (%[wq1])\n" + + "vle8.v v10, (%[wp2])\n" + "vle8.v v11, (%[wq2])\n" + "vxor.vv v10, v10, v8\n" + "vxor.vv v11, v11, v9\n" + "vse8.v v10, (%[wp2])\n" + "vse8.v v11, (%[wq2])\n" + + "vle8.v v14, (%[wp3])\n" + "vle8.v v15, (%[wq3])\n" + "vxor.vv v14, v14, v12\n" + "vxor.vv v15, v15, v13\n" + "vse8.v v14, (%[wp3])\n" + "vse8.v v15, (%[wq3])\n" + + "vle8.v v18, (%[wp4])\n" + "vle8.v v19, (%[wq4])\n" + "vxor.vv v18, v18, v16\n" + "vxor.vv v19, v19, v17\n" + "vse8.v v18, (%[wp4])\n" + "vse8.v v19, (%[wq4])\n" + + "vle8.v v22, (%[wp5])\n" + "vle8.v v23, (%[wq5])\n" + "vxor.vv v22, v22, v20\n" + "vxor.vv v23, v23, v21\n" + "vse8.v v22, (%[wp5])\n" + "vse8.v v23, (%[wq5])\n" + + "vle8.v v26, (%[wp6])\n" + "vle8.v v27, (%[wq6])\n" + "vxor.vv v26, v26, v24\n" + "vxor.vv v27, v27, v25\n" + "vse8.v v26, (%[wp6])\n" + "vse8.v v27, (%[wq6])\n" + + "vle8.v v30, (%[wp7])\n" + "vle8.v v31, (%[wq7])\n" + "vxor.vv v30, v30, v28\n" + "vxor.vv v31, v31, v29\n" + "vse8.v v30, (%[wp7])\n" + "vse8.v v31, (%[wq7])\n" + ".option pop\n" + : : + [wp0]"r"(&p[d + NSIZE * 0]), + [wq0]"r"(&q[d + NSIZE * 0]), + [wp1]"r"(&p[d + NSIZE * 1]), + [wq1]"r"(&q[d + NSIZE * 1]), + [wp2]"r"(&p[d + NSIZE * 2]), + [wq2]"r"(&q[d + NSIZE * 2]), + [wp3]"r"(&p[d + NSIZE * 3]), + [wq3]"r"(&q[d + NSIZE * 3]), + [wp4]"r"(&p[d + NSIZE * 4]), + [wq4]"r"(&q[d + NSIZE * 4]), + [wp5]"r"(&p[d + NSIZE * 5]), + [wq5]"r"(&q[d + NSIZE * 5]), + [wp6]"r"(&p[d + NSIZE * 6]), + [wq6]"r"(&q[d + NSIZE * 6]), + [wp7]"r"(&p[d + NSIZE * 7]), + [wq7]"r"(&q[d + NSIZE * 7]) + ); + } +} + +RAID6_RVV_WRAPPER(1); +RAID6_RVV_WRAPPER(2); +RAID6_RVV_WRAPPER(4); +RAID6_RVV_WRAPPER(8); diff --git a/lib/raid6/rvv.h b/lib/raid6/rvv.h new file mode 100644 index 000000000000..94044a1b707b --- /dev/null +++ b/lib/raid6/rvv.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright 2024 Institute of Software, CAS. + * + * raid6/rvv.h + * + * Definitions for RISC-V RAID-6 code + */ + +#define RAID6_RVV_WRAPPER(_n) \ + static void raid6_rvv ## _n ## _gen_syndrome(int disks, \ + size_t bytes, void **ptrs) \ + { \ + void raid6_rvv ## _n ## _gen_syndrome_real(int d, \ + unsigned long b, void **p); \ + kernel_vector_begin(); \ + raid6_rvv ## _n ## _gen_syndrome_real(disks, \ + (unsigned long)bytes, ptrs); \ + kernel_vector_end(); \ + } \ + static void raid6_rvv ## _n ## _xor_syndrome(int disks, \ + int start, int stop, \ + size_t bytes, void **ptrs) \ + { \ + void raid6_rvv ## _n ## _xor_syndrome_real(int d, \ + int s1, int s2, \ + unsigned long b, void **p); \ + kernel_vector_begin(); \ + raid6_rvv ## _n ## _xor_syndrome_real(disks, \ + start, stop, (unsigned long)bytes, ptrs); \ + kernel_vector_end(); \ + } \ + struct raid6_calls const raid6_rvvx ## _n = { \ + raid6_rvv ## _n ## _gen_syndrome, \ + raid6_rvv ## _n ## _xor_syndrome, \ + rvv_has_vector, \ + "rvvx" #_n, \ + 0 \ + } diff --git a/mm/Kconfig b/mm/Kconfig index f8bb8f070d0d..781be3240e21 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1355,6 +1355,7 @@ config NUMA_MEMBLKS config NUMA_EMU bool "NUMA emulation" depends on NUMA_MEMBLKS + depends on X86 || GENERIC_ARCH_NUMA help Enable NUMA emulation. A flat machine will be split into virtual nodes when booted with "numa=fake=N", where N is the diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index 551745df011b..c93d0c56b963 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -4,7 +4,6 @@ menu "Data Access Monitoring" config DAMON bool "DAMON: Data Access Monitoring Framework" - default y help This builds a framework that allows kernel subsystems to monitor access frequency of each memory region. The information can be useful diff --git a/mm/damon/modules-common.c b/mm/damon/modules-common.c index 7cf96574cde7..86d58f8c4f63 100644 --- a/mm/damon/modules-common.c +++ b/mm/damon/modules-common.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Common Primitives for DAMON Modules + * Common Code for DAMON Modules * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/modules-common.h b/mm/damon/modules-common.h index f49cdb417005..f103ad556368 100644 --- a/mm/damon/modules-common.h +++ b/mm/damon/modules-common.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Common Primitives for DAMON Modules + * Common Code for DAMON Modules * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 0db1fc70c84d..b43620fee6bb 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Common Primitives for Data Access Monitoring + * Common Code for Data Access Monitoring * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h index 18d837d11bce..cc9f5da9c012 100644 --- a/mm/damon/ops-common.h +++ b/mm/damon/ops-common.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Common Primitives for Data Access Monitoring + * Common Code for Data Access Monitoring * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index e8464f7e0014..4102a8c5f992 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * DAMON Primitives for The Physical Address Space + * DAMON Code for The Physical Address Space * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/sysfs-common.c b/mm/damon/sysfs-common.c index 70edf45c2174..ffaf285e241a 100644 --- a/mm/damon/sysfs-common.c +++ b/mm/damon/sysfs-common.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Common Primitives for DAMON Sysfs Interface + * Common Code for DAMON Sysfs Interface * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/sysfs-common.h b/mm/damon/sysfs-common.h index 70d84bdc9f5f..2099adee11d0 100644 --- a/mm/damon/sysfs-common.h +++ b/mm/damon/sysfs-common.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Common Primitives for DAMON Sysfs Interface + * Common Code for DAMON Sysfs Interface * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e6d99106a7f9..46554e49a478 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * DAMON Primitives for Virtual Address Spaces + * DAMON Code for Virtual Address Spaces * * Author: SeongJae Park <sj@kernel.org> */ diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f0b1d53079f9..8746ed2fec13 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -121,7 +121,7 @@ static void hugetlb_vma_lock_free(struct vm_area_struct *vma); static void hugetlb_vma_lock_alloc(struct vm_area_struct *vma); static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma); static void hugetlb_unshare_pmds(struct vm_area_struct *vma, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end, bool take_locks); static struct resv_map *vma_resv_map(struct vm_area_struct *vma); static void hugetlb_free_folio(struct folio *folio) @@ -5426,26 +5426,40 @@ static int hugetlb_vm_op_split(struct vm_area_struct *vma, unsigned long addr) { if (addr & ~(huge_page_mask(hstate_vma(vma)))) return -EINVAL; + return 0; +} +void hugetlb_split(struct vm_area_struct *vma, unsigned long addr) +{ /* * PMD sharing is only possible for PUD_SIZE-aligned address ranges * in HugeTLB VMAs. If we will lose PUD_SIZE alignment due to this * split, unshare PMDs in the PUD_SIZE interval surrounding addr now. + * This function is called in the middle of a VMA split operation, with + * MM, VMA and rmap all write-locked to prevent concurrent page table + * walks (except hardware and gup_fast()). */ + vma_assert_write_locked(vma); + i_mmap_assert_write_locked(vma->vm_file->f_mapping); + if (addr & ~PUD_MASK) { - /* - * hugetlb_vm_op_split is called right before we attempt to - * split the VMA. We will need to unshare PMDs in the old and - * new VMAs, so let's unshare before we split. - */ unsigned long floor = addr & PUD_MASK; unsigned long ceil = floor + PUD_SIZE; - if (floor >= vma->vm_start && ceil <= vma->vm_end) - hugetlb_unshare_pmds(vma, floor, ceil); + if (floor >= vma->vm_start && ceil <= vma->vm_end) { + /* + * Locking: + * Use take_locks=false here. + * The file rmap lock is already held. + * The hugetlb VMA lock can't be taken when we already + * hold the file rmap lock, and we don't need it because + * its purpose is to synchronize against concurrent page + * table walks, which are not possible thanks to the + * locks held by our caller. + */ + hugetlb_unshare_pmds(vma, floor, ceil, /* take_locks = */ false); + } } - - return 0; } static unsigned long hugetlb_vm_op_pagesize(struct vm_area_struct *vma) @@ -7615,6 +7629,13 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, return 0; pud_clear(pud); + /* + * Once our caller drops the rmap lock, some other process might be + * using this page table as a normal, non-hugetlb page table. + * Wait for pending gup_fast() in other threads to finish before letting + * that happen. + */ + tlb_remove_table_sync_one(); ptdesc_pmd_pts_dec(virt_to_ptdesc(ptep)); mm_dec_nr_pmds(mm); return 1; @@ -7885,9 +7906,16 @@ void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio, int re spin_unlock_irq(&hugetlb_lock); } +/* + * If @take_locks is false, the caller must ensure that no concurrent page table + * access can happen (except for gup_fast() and hardware page walks). + * If @take_locks is true, we take the hugetlb VMA lock (to lock out things like + * concurrent page fault handling) and the file rmap lock. + */ static void hugetlb_unshare_pmds(struct vm_area_struct *vma, unsigned long start, - unsigned long end) + unsigned long end, + bool take_locks) { struct hstate *h = hstate_vma(vma); unsigned long sz = huge_page_size(h); @@ -7911,8 +7939,12 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma, mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm, start, end); mmu_notifier_invalidate_range_start(&range); - hugetlb_vma_lock_write(vma); - i_mmap_lock_write(vma->vm_file->f_mapping); + if (take_locks) { + hugetlb_vma_lock_write(vma); + i_mmap_lock_write(vma->vm_file->f_mapping); + } else { + i_mmap_assert_write_locked(vma->vm_file->f_mapping); + } for (address = start; address < end; address += PUD_SIZE) { ptep = hugetlb_walk(vma, address, sz); if (!ptep) @@ -7922,8 +7954,10 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma, spin_unlock(ptl); } flush_hugetlb_tlb_range(vma, start, end); - i_mmap_unlock_write(vma->vm_file->f_mapping); - hugetlb_vma_unlock_write(vma); + if (take_locks) { + i_mmap_unlock_write(vma->vm_file->f_mapping); + hugetlb_vma_unlock_write(vma); + } /* * No need to call mmu_notifier_arch_invalidate_secondary_tlbs(), see * Documentation/mm/mmu_notifier.rst. @@ -7938,7 +7972,8 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma, void hugetlb_unshare_all_pmds(struct vm_area_struct *vma) { hugetlb_unshare_pmds(vma, ALIGN(vma->vm_start, PUD_SIZE), - ALIGN_DOWN(vma->vm_end, PUD_SIZE)); + ALIGN_DOWN(vma->vm_end, PUD_SIZE), + /* take_locks = */ true); } /* diff --git a/mm/kmsan/kmsan_test.c b/mm/kmsan/kmsan_test.c index 9733a22c46c1..c6c5b2bbede0 100644 --- a/mm/kmsan/kmsan_test.c +++ b/mm/kmsan/kmsan_test.c @@ -732,3 +732,4 @@ kunit_test_suites(&kmsan_test_suite); MODULE_LICENSE("GPL"); MODULE_AUTHOR("Alexander Potapenko <glider@google.com>"); +MODULE_DESCRIPTION("Test cases for KMSAN"); diff --git a/mm/madvise.c b/mm/madvise.c index 8433ac9b27e0..5f7a66a1617e 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1881,7 +1881,9 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, /* Drop and reacquire lock to unwind race. */ madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); - madvise_lock(mm, behavior); + ret = madvise_lock(mm, behavior); + if (ret) + goto out; madvise_init_tlb(&madv_behavior, mm); continue; } @@ -1892,6 +1894,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); +out: ret = (total_len - iov_iter_count(iter)) ? : ret; return ret; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 72fd72e156b1..3b1dfd08338b 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -3708,15 +3708,13 @@ static void wi_state_free(void) lockdep_is_held(&wi_state_lock)); if (!old_wi_state) { mutex_unlock(&wi_state_lock); - goto out; + return; } rcu_assign_pointer(wi_state, NULL); mutex_unlock(&wi_state_lock); synchronize_rcu(); kfree(old_wi_state); -out: - kfree(&wi_group->wi_kobj); } static struct kobj_attribute wi_auto_attr = diff --git a/mm/mremap.c b/mm/mremap.c index 83e359754961..60f6b8d0d5f0 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -237,6 +237,8 @@ static int move_ptes(struct pagetable_move_control *pmc, for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE, new_pte++, new_addr += PAGE_SIZE) { + VM_WARN_ON_ONCE(!pte_none(*new_pte)); + if (pte_none(ptep_get(old_pte))) continue; diff --git a/mm/page-writeback.c b/mm/page-writeback.c index b8eea5b3c064..72b0ff0d4bae 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -608,7 +608,7 @@ EXPORT_SYMBOL_GPL(wb_writeout_inc); */ static void writeout_period(struct timer_list *t) { - struct wb_domain *dom = from_timer(dom, t, period_timer); + struct wb_domain *dom = timer_container_of(dom, t, period_timer); int miss_periods = (jiffies - dom->period_time) / VM_COMPLETIONS_PERIOD_LEN; @@ -2203,7 +2203,7 @@ static int dirty_writeback_centisecs_handler(const struct ctl_table *table, int void laptop_mode_timer_fn(struct timer_list *t) { struct backing_dev_info *backing_dev_info = - from_timer(backing_dev_info, t, laptop_mode_wb_timer); + timer_container_of(backing_dev_info, t, laptop_mode_wb_timer); wakeup_flusher_threads_bdi(backing_dev_info, WB_REASON_LAPTOP_TIMER); } @@ -169,6 +169,9 @@ static void init_multi_vma_prep(struct vma_prepare *vp, vp->file = vma->vm_file; if (vp->file) vp->mapping = vma->vm_file->f_mapping; + + if (vmg && vmg->skip_vma_uprobe) + vp->skip_vma_uprobe = true; } /* @@ -358,10 +361,13 @@ static void vma_complete(struct vma_prepare *vp, struct vma_iterator *vmi, if (vp->file) { i_mmap_unlock_write(vp->mapping); - uprobe_mmap(vp->vma); - if (vp->adj_next) - uprobe_mmap(vp->adj_next); + if (!vp->skip_vma_uprobe) { + uprobe_mmap(vp->vma); + + if (vp->adj_next) + uprobe_mmap(vp->adj_next); + } } if (vp->remove) { @@ -539,7 +545,14 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, init_vma_prep(&vp, vma); vp.insert = new; vma_prepare(&vp); + + /* + * Get rid of huge pages and shared page tables straddling the split + * boundary. + */ vma_adjust_trans_huge(vma, vma->vm_start, addr, NULL); + if (is_vm_hugetlb_page(vma)) + hugetlb_split(vma, addr); if (new_below) { vma->vm_start = addr; @@ -1823,6 +1836,14 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, faulted_in_anon_vma = false; } + /* + * If the VMA we are copying might contain a uprobe PTE, ensure + * that we do not establish one upon merge. Otherwise, when mremap() + * moves page tables, it will orphan the newly created PTE. + */ + if (vma->vm_file) + vmg.skip_vma_uprobe = true; + new_vma = find_vma_prev(mm, addr, &vmg.prev); if (new_vma && new_vma->vm_start < addr + len) return NULL; /* should never get here */ @@ -19,6 +19,8 @@ struct vma_prepare { struct vm_area_struct *insert; struct vm_area_struct *remove; struct vm_area_struct *remove2; + + bool skip_vma_uprobe :1; }; struct unlink_vma_file_batch { @@ -120,6 +122,11 @@ struct vma_merge_struct { */ bool give_up_on_oom :1; + /* + * If set, skip uprobe_mmap upon merged vma. + */ + bool skip_vma_uprobe :1; + /* Internal flags set during merge process: */ /* diff --git a/mm/vmstat.c b/mm/vmstat.c index 6f740f070b3d..429ae5339bfe 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1201,7 +1201,6 @@ const char * const vmstat_text[] = { "nr_zone_unevictable", "nr_zone_write_pending", "nr_mlock", - "nr_bounce", #if IS_ENABLED(CONFIG_ZSMALLOC) "nr_zspages", #endif diff --git a/net/802/garp.c b/net/802/garp.c index 27f0ab146026..2d1ffc4d9462 100644 --- a/net/802/garp.c +++ b/net/802/garp.c @@ -414,7 +414,7 @@ static void garp_join_timer_arm(struct garp_applicant *app) static void garp_join_timer(struct timer_list *t) { - struct garp_applicant *app = from_timer(app, t, join_timer); + struct garp_applicant *app = timer_container_of(app, t, join_timer); spin_lock(&app->lock); garp_gid_event(app, GARP_EVENT_TRANSMIT_PDU); diff --git a/net/802/mrp.c b/net/802/mrp.c index e0c96d0da8d5..23a88305f900 100644 --- a/net/802/mrp.c +++ b/net/802/mrp.c @@ -599,7 +599,7 @@ static void mrp_join_timer_arm(struct mrp_applicant *app) static void mrp_join_timer(struct timer_list *t) { - struct mrp_applicant *app = from_timer(app, t, join_timer); + struct mrp_applicant *app = timer_container_of(app, t, join_timer); spin_lock(&app->lock); mrp_mad_event(app, MRP_EVENT_TX); @@ -621,7 +621,7 @@ static void mrp_periodic_timer_arm(struct mrp_applicant *app) static void mrp_periodic_timer(struct timer_list *t) { - struct mrp_applicant *app = from_timer(app, t, periodic_timer); + struct mrp_applicant *app = timer_container_of(app, t, periodic_timer); spin_lock(&app->lock); if (likely(app->active)) { diff --git a/net/appletalk/ddp.c b/net/appletalk/ddp.c index b068651984fe..73ea7e67f05a 100644 --- a/net/appletalk/ddp.c +++ b/net/appletalk/ddp.c @@ -169,7 +169,7 @@ found: static void atalk_destroy_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); if (sk_has_allocations(sk)) { sk->sk_timer.expires = jiffies + SOCK_DESTROY_TIME; diff --git a/net/atm/lec.c b/net/atm/lec.c index ded2f0df2ee6..acef984f3367 100644 --- a/net/atm/lec.c +++ b/net/atm/lec.c @@ -1551,7 +1551,7 @@ static void lec_arp_expire_arp(struct timer_list *t) { struct lec_arp_table *entry; - entry = from_timer(entry, t, timer); + entry = timer_container_of(entry, t, timer); pr_debug("\n"); if (entry->status == ESI_ARP_PENDING) { @@ -1572,7 +1572,8 @@ static void lec_arp_expire_arp(struct timer_list *t) static void lec_arp_expire_vcc(struct timer_list *t) { unsigned long flags; - struct lec_arp_table *to_remove = from_timer(to_remove, t, timer); + struct lec_arp_table *to_remove = timer_container_of(to_remove, t, + timer); struct lec_priv *priv = to_remove->priv; timer_delete(&to_remove->timer); diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c index b790bb92ed1c..6ef8b2a57a9b 100644 --- a/net/ax25/af_ax25.c +++ b/net/ax25/af_ax25.c @@ -287,7 +287,7 @@ void ax25_destroy_socket(ax25_cb *); */ static void ax25_destroy_timer(struct timer_list *t) { - ax25_cb *ax25 = from_timer(ax25, t, dtimer); + ax25_cb *ax25 = timer_container_of(ax25, t, dtimer); struct sock *sk; sk=ax25->sk; diff --git a/net/ax25/ax25_ds_timer.c b/net/ax25/ax25_ds_timer.c index 8d9fba069001..0c9e7775aa54 100644 --- a/net/ax25/ax25_ds_timer.c +++ b/net/ax25/ax25_ds_timer.c @@ -64,7 +64,7 @@ void ax25_ds_set_timer(ax25_dev *ax25_dev) static void ax25_ds_timeout(struct timer_list *t) { - ax25_dev *ax25_dev = from_timer(ax25_dev, t, dama.slave_timer); + ax25_dev *ax25_dev = timer_container_of(ax25_dev, t, dama.slave_timer); ax25_cb *ax25; if (ax25_dev == NULL || !ax25_dev->dama.slave) diff --git a/net/ax25/ax25_timer.c b/net/ax25/ax25_timer.c index 3891a3923d6c..a69bfbc8b679 100644 --- a/net/ax25/ax25_timer.c +++ b/net/ax25/ax25_timer.c @@ -121,7 +121,7 @@ EXPORT_SYMBOL(ax25_display_timer); static void ax25_heartbeat_expiry(struct timer_list *t) { int proto = AX25_PROTO_STD_SIMPLEX; - ax25_cb *ax25 = from_timer(ax25, t, timer); + ax25_cb *ax25 = timer_container_of(ax25, t, timer); if (ax25->ax25_dev) proto = ax25->ax25_dev->values[AX25_VALUES_PROTOCOL]; @@ -145,7 +145,7 @@ static void ax25_heartbeat_expiry(struct timer_list *t) static void ax25_t1timer_expiry(struct timer_list *t) { - ax25_cb *ax25 = from_timer(ax25, t, t1timer); + ax25_cb *ax25 = timer_container_of(ax25, t, t1timer); switch (ax25->ax25_dev->values[AX25_VALUES_PROTOCOL]) { case AX25_PROTO_STD_SIMPLEX: @@ -164,7 +164,7 @@ static void ax25_t1timer_expiry(struct timer_list *t) static void ax25_t2timer_expiry(struct timer_list *t) { - ax25_cb *ax25 = from_timer(ax25, t, t2timer); + ax25_cb *ax25 = timer_container_of(ax25, t, t2timer); switch (ax25->ax25_dev->values[AX25_VALUES_PROTOCOL]) { case AX25_PROTO_STD_SIMPLEX: @@ -183,7 +183,7 @@ static void ax25_t2timer_expiry(struct timer_list *t) static void ax25_t3timer_expiry(struct timer_list *t) { - ax25_cb *ax25 = from_timer(ax25, t, t3timer); + ax25_cb *ax25 = timer_container_of(ax25, t, t3timer); switch (ax25->ax25_dev->values[AX25_VALUES_PROTOCOL]) { case AX25_PROTO_STD_SIMPLEX: @@ -204,7 +204,7 @@ static void ax25_t3timer_expiry(struct timer_list *t) static void ax25_idletimer_expiry(struct timer_list *t) { - ax25_cb *ax25 = from_timer(ax25, t, idletimer); + ax25_cb *ax25 = timer_container_of(ax25, t, idletimer); switch (ax25->ax25_dev->values[AX25_VALUES_PROTOCOL]) { case AX25_PROTO_STD_SIMPLEX: diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c index adbadb436033..350b149e48be 100644 --- a/net/batman-adv/tp_meter.c +++ b/net/batman-adv/tp_meter.c @@ -485,7 +485,7 @@ static void batadv_tp_reset_sender_timer(struct batadv_tp_vars *tp_vars) */ static void batadv_tp_sender_timeout(struct timer_list *t) { - struct batadv_tp_vars *tp_vars = from_timer(tp_vars, t, timer); + struct batadv_tp_vars *tp_vars = timer_container_of(tp_vars, t, timer); struct batadv_priv *bat_priv = tp_vars->bat_priv; if (atomic_read(&tp_vars->sending) == 0) @@ -1101,7 +1101,7 @@ static void batadv_tp_reset_receiver_timer(struct batadv_tp_vars *tp_vars) */ static void batadv_tp_receiver_shutdown(struct timer_list *t) { - struct batadv_tp_vars *tp_vars = from_timer(tp_vars, t, timer); + struct batadv_tp_vars *tp_vars = timer_container_of(tp_vars, t, timer); struct batadv_tp_unacked *un, *safe; struct batadv_priv *bat_priv; diff --git a/net/bluetooth/eir.c b/net/bluetooth/eir.c index 1bc51e2b05a3..3f72111ba651 100644 --- a/net/bluetooth/eir.c +++ b/net/bluetooth/eir.c @@ -242,7 +242,7 @@ u8 eir_create_per_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr) return ad_len; } -u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr) +u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr, u8 size) { struct adv_info *adv = NULL; u8 ad_len = 0, flags = 0; @@ -286,7 +286,7 @@ u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr) /* If flags would still be empty, then there is no need to * include the "Flags" AD field". */ - if (flags) { + if (flags && (ad_len + eir_precalc_len(1) <= size)) { ptr[0] = 0x02; ptr[1] = EIR_FLAGS; ptr[2] = flags; @@ -316,7 +316,8 @@ skip_flags: } /* Provide Tx Power only if we can provide a valid value for it */ - if (adv_tx_power != HCI_TX_POWER_INVALID) { + if (adv_tx_power != HCI_TX_POWER_INVALID && + (ad_len + eir_precalc_len(1) <= size)) { ptr[0] = 0x02; ptr[1] = EIR_TX_POWER; ptr[2] = (u8)adv_tx_power; @@ -366,17 +367,19 @@ u8 eir_create_scan_rsp(struct hci_dev *hdev, u8 instance, u8 *ptr) void *eir_get_service_data(u8 *eir, size_t eir_len, u16 uuid, size_t *len) { - while ((eir = eir_get_data(eir, eir_len, EIR_SERVICE_DATA, len))) { + size_t dlen; + + while ((eir = eir_get_data(eir, eir_len, EIR_SERVICE_DATA, &dlen))) { u16 value = get_unaligned_le16(eir); if (uuid == value) { if (len) - *len -= 2; + *len = dlen - 2; return &eir[2]; } - eir += *len; - eir_len -= *len; + eir += dlen; + eir_len -= dlen; } return NULL; diff --git a/net/bluetooth/eir.h b/net/bluetooth/eir.h index 5c89a05e8b29..9372db83f912 100644 --- a/net/bluetooth/eir.h +++ b/net/bluetooth/eir.h @@ -9,7 +9,7 @@ void eir_create(struct hci_dev *hdev, u8 *data); -u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr); +u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr, u8 size); u8 eir_create_scan_rsp(struct hci_dev *hdev, u8 instance, u8 *ptr); u8 eir_create_per_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr); diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c index 99efeed6a766..4f379184df5b 100644 --- a/net/bluetooth/hci_conn.c +++ b/net/bluetooth/hci_conn.c @@ -1501,8 +1501,8 @@ static int qos_set_bis(struct hci_dev *hdev, struct bt_iso_qos *qos) /* This function requires the caller holds hdev->lock */ static struct hci_conn *hci_add_bis(struct hci_dev *hdev, bdaddr_t *dst, - struct bt_iso_qos *qos, __u8 base_len, - __u8 *base) + __u8 sid, struct bt_iso_qos *qos, + __u8 base_len, __u8 *base) { struct hci_conn *conn; int err; @@ -1543,6 +1543,7 @@ static struct hci_conn *hci_add_bis(struct hci_dev *hdev, bdaddr_t *dst, return conn; conn->state = BT_CONNECT; + conn->sid = sid; hci_conn_hold(conn); return conn; @@ -2062,7 +2063,8 @@ static int create_big_sync(struct hci_dev *hdev, void *data) if (qos->bcast.bis) sync_interval = interval * 4; - err = hci_start_per_adv_sync(hdev, qos->bcast.bis, conn->le_per_adv_data_len, + err = hci_start_per_adv_sync(hdev, qos->bcast.bis, conn->sid, + conn->le_per_adv_data_len, conn->le_per_adv_data, flags, interval, interval, sync_interval); if (err) @@ -2134,7 +2136,7 @@ static void create_big_complete(struct hci_dev *hdev, void *data, int err) } } -struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, +struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, __u8 sid, struct bt_iso_qos *qos, __u8 base_len, __u8 *base) { @@ -2156,7 +2158,7 @@ struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, base, base_len); /* We need hci_conn object using the BDADDR_ANY as dst */ - conn = hci_add_bis(hdev, dst, qos, base_len, eir); + conn = hci_add_bis(hdev, dst, sid, qos, base_len, eir); if (IS_ERR(conn)) return conn; @@ -2207,20 +2209,35 @@ static void bis_mark_per_adv(struct hci_conn *conn, void *data) } struct hci_conn *hci_connect_bis(struct hci_dev *hdev, bdaddr_t *dst, - __u8 dst_type, struct bt_iso_qos *qos, + __u8 dst_type, __u8 sid, + struct bt_iso_qos *qos, __u8 base_len, __u8 *base) { struct hci_conn *conn; int err; struct iso_list_data data; - conn = hci_bind_bis(hdev, dst, qos, base_len, base); + conn = hci_bind_bis(hdev, dst, sid, qos, base_len, base); if (IS_ERR(conn)) return conn; if (conn->state == BT_CONNECTED) return conn; + /* Check if SID needs to be allocated then search for the first + * available. + */ + if (conn->sid == HCI_SID_INVALID) { + u8 sid; + + for (sid = 0; sid <= 0x0f; sid++) { + if (!hci_find_adv_sid(hdev, sid)) { + conn->sid = sid; + break; + } + } + } + data.big = qos->bcast.big; data.bis = qos->bcast.bis; diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c index 3b49828160b7..07a8b4281a39 100644 --- a/net/bluetooth/hci_core.c +++ b/net/bluetooth/hci_core.c @@ -1585,6 +1585,19 @@ struct adv_info *hci_find_adv_instance(struct hci_dev *hdev, u8 instance) } /* This function requires the caller holds hdev->lock */ +struct adv_info *hci_find_adv_sid(struct hci_dev *hdev, u8 sid) +{ + struct adv_info *adv; + + list_for_each_entry(adv, &hdev->adv_instances, list) { + if (adv->sid == sid) + return adv; + } + + return NULL; +} + +/* This function requires the caller holds hdev->lock */ struct adv_info *hci_get_next_instance(struct hci_dev *hdev, u8 instance) { struct adv_info *cur_instance; @@ -1736,7 +1749,7 @@ struct adv_info *hci_add_adv_instance(struct hci_dev *hdev, u8 instance, } /* This function requires the caller holds hdev->lock */ -struct adv_info *hci_add_per_instance(struct hci_dev *hdev, u8 instance, +struct adv_info *hci_add_per_instance(struct hci_dev *hdev, u8 instance, u8 sid, u32 flags, u8 data_len, u8 *data, u32 min_interval, u32 max_interval) { @@ -1748,6 +1761,7 @@ struct adv_info *hci_add_per_instance(struct hci_dev *hdev, u8 instance, if (IS_ERR(adv)) return adv; + adv->sid = sid; adv->periodic = true; adv->per_adv_data_len = data_len; @@ -1877,10 +1891,8 @@ void hci_free_adv_monitor(struct hci_dev *hdev, struct adv_monitor *monitor) if (monitor->handle) idr_remove(&hdev->adv_monitors_idr, monitor->handle); - if (monitor->state != ADV_MONITOR_STATE_NOT_REGISTERED) { + if (monitor->state != ADV_MONITOR_STATE_NOT_REGISTERED) hdev->adv_monitors_cnt--; - mgmt_adv_monitor_removed(hdev, monitor->handle); - } kfree(monitor); } @@ -2487,6 +2499,7 @@ struct hci_dev *hci_alloc_dev_priv(int sizeof_priv) mutex_init(&hdev->lock); mutex_init(&hdev->req_lock); + mutex_init(&hdev->mgmt_pending_lock); ida_init(&hdev->unset_handle_ida); @@ -3417,23 +3430,18 @@ static void hci_link_tx_to(struct hci_dev *hdev, __u8 type) bt_dev_err(hdev, "link tx timeout"); - rcu_read_lock(); + hci_dev_lock(hdev); /* Kill stalled connections */ - list_for_each_entry_rcu(c, &h->list, list) { + list_for_each_entry(c, &h->list, list) { if (c->type == type && c->sent) { bt_dev_err(hdev, "killing stalled connection %pMR", &c->dst); - /* hci_disconnect might sleep, so, we have to release - * the RCU read lock before calling it. - */ - rcu_read_unlock(); hci_disconnect(c, HCI_ERROR_REMOTE_USER_TERM); - rcu_read_lock(); } } - rcu_read_unlock(); + hci_dev_unlock(hdev); } static struct hci_chan *hci_chan_sent(struct hci_dev *hdev, __u8 type, diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index 62d1ff951ebe..6687f2a4d1eb 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -1261,10 +1261,12 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) hci_cpu_to_le24(adv->min_interval, cp.min_interval); hci_cpu_to_le24(adv->max_interval, cp.max_interval); cp.tx_power = adv->tx_power; + cp.sid = adv->sid; } else { hci_cpu_to_le24(hdev->le_adv_min_interval, cp.min_interval); hci_cpu_to_le24(hdev->le_adv_max_interval, cp.max_interval); cp.tx_power = HCI_ADV_TX_POWER_NO_PREFERENCE; + cp.sid = 0x00; } secondary_adv = (flags & MGMT_ADV_FLAG_SEC_MASK); @@ -1559,7 +1561,8 @@ static int hci_enable_per_advertising_sync(struct hci_dev *hdev, u8 instance) static int hci_adv_bcast_annoucement(struct hci_dev *hdev, struct adv_info *adv) { u8 bid[3]; - u8 ad[4 + 3]; + u8 ad[HCI_MAX_EXT_AD_LENGTH]; + u8 len; /* Skip if NULL adv as instance 0x00 is used for general purpose * advertising so it cannot used for the likes of Broadcast Announcement @@ -1585,14 +1588,16 @@ static int hci_adv_bcast_annoucement(struct hci_dev *hdev, struct adv_info *adv) /* Generate Broadcast ID */ get_random_bytes(bid, sizeof(bid)); - eir_append_service_data(ad, 0, 0x1852, bid, sizeof(bid)); - hci_set_adv_instance_data(hdev, adv->instance, sizeof(ad), ad, 0, NULL); + len = eir_append_service_data(ad, 0, 0x1852, bid, sizeof(bid)); + memcpy(ad + len, adv->adv_data, adv->adv_data_len); + hci_set_adv_instance_data(hdev, adv->instance, len + adv->adv_data_len, + ad, 0, NULL); return hci_update_adv_data_sync(hdev, adv->instance); } -int hci_start_per_adv_sync(struct hci_dev *hdev, u8 instance, u8 data_len, - u8 *data, u32 flags, u16 min_interval, +int hci_start_per_adv_sync(struct hci_dev *hdev, u8 instance, u8 sid, + u8 data_len, u8 *data, u32 flags, u16 min_interval, u16 max_interval, u16 sync_interval) { struct adv_info *adv = NULL; @@ -1603,9 +1608,28 @@ int hci_start_per_adv_sync(struct hci_dev *hdev, u8 instance, u8 data_len, if (instance) { adv = hci_find_adv_instance(hdev, instance); - /* Create an instance if that could not be found */ - if (!adv) { - adv = hci_add_per_instance(hdev, instance, flags, + if (adv) { + if (sid != HCI_SID_INVALID && adv->sid != sid) { + /* If the SID don't match attempt to find by + * SID. + */ + adv = hci_find_adv_sid(hdev, sid); + if (!adv) { + bt_dev_err(hdev, + "Unable to find adv_info"); + return -EINVAL; + } + } + + /* Turn it into periodic advertising */ + adv->periodic = true; + adv->per_adv_data_len = data_len; + if (data) + memcpy(adv->per_adv_data, data, data_len); + adv->flags = flags; + } else if (!adv) { + /* Create an instance if that could not be found */ + adv = hci_add_per_instance(hdev, instance, sid, flags, data_len, data, sync_interval, sync_interval); @@ -1812,7 +1836,8 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) return 0; } - len = eir_create_adv_data(hdev, instance, pdu->data); + len = eir_create_adv_data(hdev, instance, pdu->data, + HCI_MAX_EXT_AD_LENGTH); pdu->length = len; pdu->handle = adv ? adv->handle : instance; @@ -1843,7 +1868,7 @@ static int hci_set_adv_data_sync(struct hci_dev *hdev, u8 instance) memset(&cp, 0, sizeof(cp)); - len = eir_create_adv_data(hdev, instance, cp.data); + len = eir_create_adv_data(hdev, instance, cp.data, sizeof(cp.data)); /* There's nothing to do if the data hasn't changed */ if (hdev->adv_data_len == len && diff --git a/net/bluetooth/hidp/core.c b/net/bluetooth/hidp/core.c index fc5af8639b1e..6724adce615b 100644 --- a/net/bluetooth/hidp/core.c +++ b/net/bluetooth/hidp/core.c @@ -405,7 +405,7 @@ static int hidp_raw_request(struct hid_device *hid, unsigned char reportnum, static void hidp_idle_timeout(struct timer_list *t) { - struct hidp_session *session = from_timer(session, t, timer); + struct hidp_session *session = timer_container_of(session, t, timer); /* The HIDP user-space API only contains calls to add and remove * devices. There is no way to forward events of any kind. Therefore, diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c index 6e2c752aaa8f..3c2c98eecc62 100644 --- a/net/bluetooth/iso.c +++ b/net/bluetooth/iso.c @@ -336,7 +336,7 @@ static int iso_connect_bis(struct sock *sk) struct hci_dev *hdev; int err; - BT_DBG("%pMR", &iso_pi(sk)->src); + BT_DBG("%pMR (SID 0x%2.2x)", &iso_pi(sk)->src, iso_pi(sk)->bc_sid); hdev = hci_get_route(&iso_pi(sk)->dst, &iso_pi(sk)->src, iso_pi(sk)->src_type); @@ -365,7 +365,7 @@ static int iso_connect_bis(struct sock *sk) /* Just bind if DEFER_SETUP has been set */ if (test_bit(BT_SK_DEFER_SETUP, &bt_sk(sk)->flags)) { - hcon = hci_bind_bis(hdev, &iso_pi(sk)->dst, + hcon = hci_bind_bis(hdev, &iso_pi(sk)->dst, iso_pi(sk)->bc_sid, &iso_pi(sk)->qos, iso_pi(sk)->base_len, iso_pi(sk)->base); if (IS_ERR(hcon)) { @@ -375,12 +375,16 @@ static int iso_connect_bis(struct sock *sk) } else { hcon = hci_connect_bis(hdev, &iso_pi(sk)->dst, le_addr_type(iso_pi(sk)->dst_type), - &iso_pi(sk)->qos, iso_pi(sk)->base_len, - iso_pi(sk)->base); + iso_pi(sk)->bc_sid, &iso_pi(sk)->qos, + iso_pi(sk)->base_len, iso_pi(sk)->base); if (IS_ERR(hcon)) { err = PTR_ERR(hcon); goto unlock; } + + /* Update SID if it was not set */ + if (iso_pi(sk)->bc_sid == HCI_SID_INVALID) + iso_pi(sk)->bc_sid = hcon->sid; } conn = iso_conn_add(hcon); @@ -1337,10 +1341,13 @@ static int iso_sock_getname(struct socket *sock, struct sockaddr *addr, addr->sa_family = AF_BLUETOOTH; if (peer) { + struct hci_conn *hcon = iso_pi(sk)->conn ? + iso_pi(sk)->conn->hcon : NULL; + bacpy(&sa->iso_bdaddr, &iso_pi(sk)->dst); sa->iso_bdaddr_type = iso_pi(sk)->dst_type; - if (test_bit(BT_SK_PA_SYNC, &iso_pi(sk)->flags)) { + if (hcon && hcon->type == BIS_LINK) { sa->iso_bc->bc_sid = iso_pi(sk)->bc_sid; sa->iso_bc->bc_num_bis = iso_pi(sk)->bc_num_bis; memcpy(sa->iso_bc->bc_bis, iso_pi(sk)->bc_bis, diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c index 14a9462fced5..d540f7b4f75f 100644 --- a/net/bluetooth/mgmt.c +++ b/net/bluetooth/mgmt.c @@ -1447,22 +1447,17 @@ static void settings_rsp(struct mgmt_pending_cmd *cmd, void *data) send_settings_rsp(cmd->sk, cmd->opcode, match->hdev); - list_del(&cmd->list); - if (match->sk == NULL) { match->sk = cmd->sk; sock_hold(match->sk); } - - mgmt_pending_free(cmd); } static void cmd_status_rsp(struct mgmt_pending_cmd *cmd, void *data) { u8 *status = data; - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, *status); - mgmt_pending_remove(cmd); + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, *status); } static void cmd_complete_rsp(struct mgmt_pending_cmd *cmd, void *data) @@ -1476,8 +1471,6 @@ static void cmd_complete_rsp(struct mgmt_pending_cmd *cmd, void *data) if (cmd->cmd_complete) { cmd->cmd_complete(cmd, match->mgmt_status); - mgmt_pending_remove(cmd); - return; } @@ -1486,13 +1479,13 @@ static void cmd_complete_rsp(struct mgmt_pending_cmd *cmd, void *data) static int generic_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status) { - return mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status, + return mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, status, cmd->param, cmd->param_len); } static int addr_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status) { - return mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status, + return mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, status, cmd->param, sizeof(struct mgmt_addr_info)); } @@ -1532,7 +1525,7 @@ static void mgmt_set_discoverable_complete(struct hci_dev *hdev, void *data, if (err) { u8 mgmt_err = mgmt_status(err); - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err); + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_err); hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE); goto done; } @@ -1707,7 +1700,7 @@ static void mgmt_set_connectable_complete(struct hci_dev *hdev, void *data, if (err) { u8 mgmt_err = mgmt_status(err); - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err); + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_err); goto done; } @@ -1943,8 +1936,8 @@ static void set_ssp_complete(struct hci_dev *hdev, void *data, int err) new_settings(hdev, NULL); } - mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, cmd_status_rsp, - &mgmt_err); + mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, true, + cmd_status_rsp, &mgmt_err); return; } @@ -1954,7 +1947,7 @@ static void set_ssp_complete(struct hci_dev *hdev, void *data, int err) changed = hci_dev_test_and_clear_flag(hdev, HCI_SSP_ENABLED); } - mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, settings_rsp, &match); + mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, true, settings_rsp, &match); if (changed) new_settings(hdev, match.sk); @@ -2074,12 +2067,12 @@ static void set_le_complete(struct hci_dev *hdev, void *data, int err) bt_dev_dbg(hdev, "err %d", err); if (status) { - mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, cmd_status_rsp, - &status); + mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, true, cmd_status_rsp, + &status); return; } - mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, settings_rsp, &match); + mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, true, settings_rsp, &match); new_settings(hdev, match.sk); @@ -2138,7 +2131,7 @@ static void set_mesh_complete(struct hci_dev *hdev, void *data, int err) struct sock *sk = cmd->sk; if (status) { - mgmt_pending_foreach(MGMT_OP_SET_MESH_RECEIVER, hdev, + mgmt_pending_foreach(MGMT_OP_SET_MESH_RECEIVER, hdev, true, cmd_status_rsp, &status); return; } @@ -2638,7 +2631,7 @@ static void mgmt_class_complete(struct hci_dev *hdev, void *data, int err) bt_dev_dbg(hdev, "err %d", err); - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), hdev->dev_class, 3); mgmt_pending_free(cmd); @@ -3427,7 +3420,7 @@ static int pairing_complete(struct mgmt_pending_cmd *cmd, u8 status) bacpy(&rp.addr.bdaddr, &conn->dst); rp.addr.type = link_to_bdaddr(conn->type, conn->dst_type); - err = mgmt_cmd_complete(cmd->sk, cmd->index, MGMT_OP_PAIR_DEVICE, + err = mgmt_cmd_complete(cmd->sk, cmd->hdev->id, MGMT_OP_PAIR_DEVICE, status, &rp, sizeof(rp)); /* So we don't get further callbacks for this connection */ @@ -5108,24 +5101,14 @@ static void mgmt_adv_monitor_added(struct sock *sk, struct hci_dev *hdev, mgmt_event(MGMT_EV_ADV_MONITOR_ADDED, hdev, &ev, sizeof(ev), sk); } -void mgmt_adv_monitor_removed(struct hci_dev *hdev, u16 handle) +static void mgmt_adv_monitor_removed(struct sock *sk, struct hci_dev *hdev, + __le16 handle) { struct mgmt_ev_adv_monitor_removed ev; - struct mgmt_pending_cmd *cmd; - struct sock *sk_skip = NULL; - struct mgmt_cp_remove_adv_monitor *cp; - - cmd = pending_find(MGMT_OP_REMOVE_ADV_MONITOR, hdev); - if (cmd) { - cp = cmd->param; - - if (cp->monitor_handle) - sk_skip = cmd->sk; - } - ev.monitor_handle = cpu_to_le16(handle); + ev.monitor_handle = handle; - mgmt_event(MGMT_EV_ADV_MONITOR_REMOVED, hdev, &ev, sizeof(ev), sk_skip); + mgmt_event(MGMT_EV_ADV_MONITOR_REMOVED, hdev, &ev, sizeof(ev), sk); } static int read_adv_mon_features(struct sock *sk, struct hci_dev *hdev, @@ -5196,7 +5179,7 @@ static void mgmt_add_adv_patterns_monitor_complete(struct hci_dev *hdev, hci_update_passive_scan(hdev); } - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(status), &rp, sizeof(rp)); mgmt_pending_remove(cmd); @@ -5227,8 +5210,7 @@ static int __add_adv_patterns_monitor(struct sock *sk, struct hci_dev *hdev, if (pending_find(MGMT_OP_SET_LE, hdev) || pending_find(MGMT_OP_ADD_ADV_PATTERNS_MONITOR, hdev) || - pending_find(MGMT_OP_ADD_ADV_PATTERNS_MONITOR_RSSI, hdev) || - pending_find(MGMT_OP_REMOVE_ADV_MONITOR, hdev)) { + pending_find(MGMT_OP_ADD_ADV_PATTERNS_MONITOR_RSSI, hdev)) { status = MGMT_STATUS_BUSY; goto unlock; } @@ -5398,8 +5380,7 @@ static void mgmt_remove_adv_monitor_complete(struct hci_dev *hdev, struct mgmt_pending_cmd *cmd = data; struct mgmt_cp_remove_adv_monitor *cp; - if (status == -ECANCELED || - cmd != pending_find(MGMT_OP_REMOVE_ADV_MONITOR, hdev)) + if (status == -ECANCELED) return; hci_dev_lock(hdev); @@ -5408,12 +5389,14 @@ static void mgmt_remove_adv_monitor_complete(struct hci_dev *hdev, rp.monitor_handle = cp->monitor_handle; - if (!status) + if (!status) { + mgmt_adv_monitor_removed(cmd->sk, hdev, cp->monitor_handle); hci_update_passive_scan(hdev); + } - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(status), &rp, sizeof(rp)); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); hci_dev_unlock(hdev); bt_dev_dbg(hdev, "remove monitor %d complete, status %d", @@ -5423,10 +5406,6 @@ static void mgmt_remove_adv_monitor_complete(struct hci_dev *hdev, static int mgmt_remove_adv_monitor_sync(struct hci_dev *hdev, void *data) { struct mgmt_pending_cmd *cmd = data; - - if (cmd != pending_find(MGMT_OP_REMOVE_ADV_MONITOR, hdev)) - return -ECANCELED; - struct mgmt_cp_remove_adv_monitor *cp = cmd->param; u16 handle = __le16_to_cpu(cp->monitor_handle); @@ -5445,14 +5424,13 @@ static int remove_adv_monitor(struct sock *sk, struct hci_dev *hdev, hci_dev_lock(hdev); if (pending_find(MGMT_OP_SET_LE, hdev) || - pending_find(MGMT_OP_REMOVE_ADV_MONITOR, hdev) || pending_find(MGMT_OP_ADD_ADV_PATTERNS_MONITOR, hdev) || pending_find(MGMT_OP_ADD_ADV_PATTERNS_MONITOR_RSSI, hdev)) { status = MGMT_STATUS_BUSY; goto unlock; } - cmd = mgmt_pending_add(sk, MGMT_OP_REMOVE_ADV_MONITOR, hdev, data, len); + cmd = mgmt_pending_new(sk, MGMT_OP_REMOVE_ADV_MONITOR, hdev, data, len); if (!cmd) { status = MGMT_STATUS_NO_RESOURCES; goto unlock; @@ -5462,7 +5440,7 @@ static int remove_adv_monitor(struct sock *sk, struct hci_dev *hdev, mgmt_remove_adv_monitor_complete); if (err) { - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); if (err == -ENOMEM) status = MGMT_STATUS_NO_RESOURCES; @@ -5792,7 +5770,7 @@ static void start_discovery_complete(struct hci_dev *hdev, void *data, int err) cmd != pending_find(MGMT_OP_START_SERVICE_DISCOVERY, hdev)) return; - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, mgmt_status(err), + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), cmd->param, 1); mgmt_pending_remove(cmd); @@ -6013,7 +5991,7 @@ static void stop_discovery_complete(struct hci_dev *hdev, void *data, int err) bt_dev_dbg(hdev, "err %d", err); - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, mgmt_status(err), + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), cmd->param, 1); mgmt_pending_remove(cmd); @@ -6238,7 +6216,7 @@ static void set_advertising_complete(struct hci_dev *hdev, void *data, int err) u8 status = mgmt_status(err); if (status) { - mgmt_pending_foreach(MGMT_OP_SET_ADVERTISING, hdev, + mgmt_pending_foreach(MGMT_OP_SET_ADVERTISING, hdev, true, cmd_status_rsp, &status); return; } @@ -6248,7 +6226,7 @@ static void set_advertising_complete(struct hci_dev *hdev, void *data, int err) else hci_dev_clear_flag(hdev, HCI_ADVERTISING); - mgmt_pending_foreach(MGMT_OP_SET_ADVERTISING, hdev, settings_rsp, + mgmt_pending_foreach(MGMT_OP_SET_ADVERTISING, hdev, true, settings_rsp, &match); new_settings(hdev, match.sk); @@ -6592,7 +6570,7 @@ static void set_bredr_complete(struct hci_dev *hdev, void *data, int err) */ hci_dev_clear_flag(hdev, HCI_BREDR_ENABLED); - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err); + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_err); } else { send_settings_rsp(cmd->sk, MGMT_OP_SET_BREDR, hdev); new_settings(hdev, cmd->sk); @@ -6729,7 +6707,7 @@ static void set_secure_conn_complete(struct hci_dev *hdev, void *data, int err) if (err) { u8 mgmt_err = mgmt_status(err); - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err); + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_err); goto done; } @@ -7176,7 +7154,7 @@ static void get_conn_info_complete(struct hci_dev *hdev, void *data, int err) rp.max_tx_power = HCI_TX_POWER_INVALID; } - mgmt_cmd_complete(cmd->sk, cmd->index, MGMT_OP_GET_CONN_INFO, status, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, MGMT_OP_GET_CONN_INFO, status, &rp, sizeof(rp)); mgmt_pending_free(cmd); @@ -7336,7 +7314,7 @@ static void get_clock_info_complete(struct hci_dev *hdev, void *data, int err) } complete: - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status, &rp, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, status, &rp, sizeof(rp)); mgmt_pending_free(cmd); @@ -8586,10 +8564,10 @@ static void add_advertising_complete(struct hci_dev *hdev, void *data, int err) rp.instance = cp->instance; if (err) - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err)); else - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), &rp, sizeof(rp)); add_adv_complete(hdev, cmd->sk, cp->instance, err); @@ -8777,10 +8755,10 @@ static void add_ext_adv_params_complete(struct hci_dev *hdev, void *data, hci_remove_adv_instance(hdev, cp->instance); - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err)); } else { - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), &rp, sizeof(rp)); } @@ -8927,10 +8905,10 @@ static void add_ext_adv_data_complete(struct hci_dev *hdev, void *data, int err) rp.instance = cp->instance; if (err) - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err)); else - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err), &rp, sizeof(rp)); mgmt_pending_free(cmd); @@ -9089,10 +9067,10 @@ static void remove_advertising_complete(struct hci_dev *hdev, void *data, rp.instance = cp->instance; if (err) - mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_status(cmd->sk, cmd->hdev->id, cmd->opcode, mgmt_status(err)); else - mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, MGMT_STATUS_SUCCESS, &rp, sizeof(rp)); mgmt_pending_free(cmd); @@ -9364,7 +9342,7 @@ void mgmt_index_removed(struct hci_dev *hdev) if (test_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks)) return; - mgmt_pending_foreach(0, hdev, cmd_complete_rsp, &match); + mgmt_pending_foreach(0, hdev, true, cmd_complete_rsp, &match); if (hci_dev_test_flag(hdev, HCI_UNCONFIGURED)) { mgmt_index_event(MGMT_EV_UNCONF_INDEX_REMOVED, hdev, NULL, 0, @@ -9402,7 +9380,8 @@ void mgmt_power_on(struct hci_dev *hdev, int err) hci_update_passive_scan(hdev); } - mgmt_pending_foreach(MGMT_OP_SET_POWERED, hdev, settings_rsp, &match); + mgmt_pending_foreach(MGMT_OP_SET_POWERED, hdev, true, settings_rsp, + &match); new_settings(hdev, match.sk); @@ -9417,7 +9396,8 @@ void __mgmt_power_off(struct hci_dev *hdev) struct cmd_lookup match = { NULL, hdev }; u8 zero_cod[] = { 0, 0, 0 }; - mgmt_pending_foreach(MGMT_OP_SET_POWERED, hdev, settings_rsp, &match); + mgmt_pending_foreach(MGMT_OP_SET_POWERED, hdev, true, settings_rsp, + &match); /* If the power off is because of hdev unregistration let * use the appropriate INVALID_INDEX status. Otherwise use @@ -9431,7 +9411,7 @@ void __mgmt_power_off(struct hci_dev *hdev) else match.mgmt_status = MGMT_STATUS_NOT_POWERED; - mgmt_pending_foreach(0, hdev, cmd_complete_rsp, &match); + mgmt_pending_foreach(0, hdev, true, cmd_complete_rsp, &match); if (memcmp(hdev->dev_class, zero_cod, sizeof(zero_cod)) != 0) { mgmt_limited_event(MGMT_EV_CLASS_OF_DEV_CHANGED, hdev, @@ -9672,7 +9652,6 @@ static void unpair_device_rsp(struct mgmt_pending_cmd *cmd, void *data) device_unpaired(hdev, &cp->addr.bdaddr, cp->addr.type, cmd->sk); cmd->cmd_complete(cmd, 0); - mgmt_pending_remove(cmd); } bool mgmt_powering_down(struct hci_dev *hdev) @@ -9728,8 +9707,8 @@ void mgmt_disconnect_failed(struct hci_dev *hdev, bdaddr_t *bdaddr, struct mgmt_cp_disconnect *cp; struct mgmt_pending_cmd *cmd; - mgmt_pending_foreach(MGMT_OP_UNPAIR_DEVICE, hdev, unpair_device_rsp, - hdev); + mgmt_pending_foreach(MGMT_OP_UNPAIR_DEVICE, hdev, true, + unpair_device_rsp, hdev); cmd = pending_find(MGMT_OP_DISCONNECT, hdev); if (!cmd) @@ -9922,7 +9901,7 @@ void mgmt_auth_enable_complete(struct hci_dev *hdev, u8 status) if (status) { u8 mgmt_err = mgmt_status(status); - mgmt_pending_foreach(MGMT_OP_SET_LINK_SECURITY, hdev, + mgmt_pending_foreach(MGMT_OP_SET_LINK_SECURITY, hdev, true, cmd_status_rsp, &mgmt_err); return; } @@ -9932,8 +9911,8 @@ void mgmt_auth_enable_complete(struct hci_dev *hdev, u8 status) else changed = hci_dev_test_and_clear_flag(hdev, HCI_LINK_SECURITY); - mgmt_pending_foreach(MGMT_OP_SET_LINK_SECURITY, hdev, settings_rsp, - &match); + mgmt_pending_foreach(MGMT_OP_SET_LINK_SECURITY, hdev, true, + settings_rsp, &match); if (changed) new_settings(hdev, match.sk); @@ -9957,9 +9936,12 @@ void mgmt_set_class_of_dev_complete(struct hci_dev *hdev, u8 *dev_class, { struct cmd_lookup match = { NULL, hdev, mgmt_status(status) }; - mgmt_pending_foreach(MGMT_OP_SET_DEV_CLASS, hdev, sk_lookup, &match); - mgmt_pending_foreach(MGMT_OP_ADD_UUID, hdev, sk_lookup, &match); - mgmt_pending_foreach(MGMT_OP_REMOVE_UUID, hdev, sk_lookup, &match); + mgmt_pending_foreach(MGMT_OP_SET_DEV_CLASS, hdev, false, sk_lookup, + &match); + mgmt_pending_foreach(MGMT_OP_ADD_UUID, hdev, false, sk_lookup, + &match); + mgmt_pending_foreach(MGMT_OP_REMOVE_UUID, hdev, false, sk_lookup, + &match); if (!status) { mgmt_limited_event(MGMT_EV_CLASS_OF_DEV_CHANGED, hdev, dev_class, diff --git a/net/bluetooth/mgmt_util.c b/net/bluetooth/mgmt_util.c index 3713ff490c65..a88a07da3947 100644 --- a/net/bluetooth/mgmt_util.c +++ b/net/bluetooth/mgmt_util.c @@ -217,30 +217,47 @@ int mgmt_cmd_complete(struct sock *sk, u16 index, u16 cmd, u8 status, struct mgmt_pending_cmd *mgmt_pending_find(unsigned short channel, u16 opcode, struct hci_dev *hdev) { - struct mgmt_pending_cmd *cmd; + struct mgmt_pending_cmd *cmd, *tmp; + + mutex_lock(&hdev->mgmt_pending_lock); - list_for_each_entry(cmd, &hdev->mgmt_pending, list) { + list_for_each_entry_safe(cmd, tmp, &hdev->mgmt_pending, list) { if (hci_sock_get_channel(cmd->sk) != channel) continue; - if (cmd->opcode == opcode) + + if (cmd->opcode == opcode) { + mutex_unlock(&hdev->mgmt_pending_lock); return cmd; + } } + mutex_unlock(&hdev->mgmt_pending_lock); + return NULL; } -void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev, +void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev, bool remove, void (*cb)(struct mgmt_pending_cmd *cmd, void *data), void *data) { struct mgmt_pending_cmd *cmd, *tmp; + mutex_lock(&hdev->mgmt_pending_lock); + list_for_each_entry_safe(cmd, tmp, &hdev->mgmt_pending, list) { if (opcode > 0 && cmd->opcode != opcode) continue; + if (remove) + list_del(&cmd->list); + cb(cmd, data); + + if (remove) + mgmt_pending_free(cmd); } + + mutex_unlock(&hdev->mgmt_pending_lock); } struct mgmt_pending_cmd *mgmt_pending_new(struct sock *sk, u16 opcode, @@ -254,7 +271,7 @@ struct mgmt_pending_cmd *mgmt_pending_new(struct sock *sk, u16 opcode, return NULL; cmd->opcode = opcode; - cmd->index = hdev->id; + cmd->hdev = hdev; cmd->param = kmemdup(data, len, GFP_KERNEL); if (!cmd->param) { @@ -280,7 +297,9 @@ struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode, if (!cmd) return NULL; + mutex_lock(&hdev->mgmt_pending_lock); list_add_tail(&cmd->list, &hdev->mgmt_pending); + mutex_unlock(&hdev->mgmt_pending_lock); return cmd; } @@ -294,7 +313,10 @@ void mgmt_pending_free(struct mgmt_pending_cmd *cmd) void mgmt_pending_remove(struct mgmt_pending_cmd *cmd) { + mutex_lock(&cmd->hdev->mgmt_pending_lock); list_del(&cmd->list); + mutex_unlock(&cmd->hdev->mgmt_pending_lock); + mgmt_pending_free(cmd); } diff --git a/net/bluetooth/mgmt_util.h b/net/bluetooth/mgmt_util.h index f2ba994ab1d8..024e51dd6937 100644 --- a/net/bluetooth/mgmt_util.h +++ b/net/bluetooth/mgmt_util.h @@ -33,7 +33,7 @@ struct mgmt_mesh_tx { struct mgmt_pending_cmd { struct list_head list; u16 opcode; - int index; + struct hci_dev *hdev; void *param; size_t param_len; struct sock *sk; @@ -54,7 +54,7 @@ int mgmt_cmd_complete(struct sock *sk, u16 index, u16 cmd, u8 status, struct mgmt_pending_cmd *mgmt_pending_find(unsigned short channel, u16 opcode, struct hci_dev *hdev); -void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev, +void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev, bool remove, void (*cb)(struct mgmt_pending_cmd *cmd, void *data), void *data); struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode, diff --git a/net/bluetooth/rfcomm/core.c b/net/bluetooth/rfcomm/core.c index 20ea7dba0a9a..3b8f39618d65 100644 --- a/net/bluetooth/rfcomm/core.c +++ b/net/bluetooth/rfcomm/core.c @@ -235,7 +235,7 @@ static int rfcomm_check_security(struct rfcomm_dlc *d) static void rfcomm_session_timeout(struct timer_list *t) { - struct rfcomm_session *s = from_timer(s, t, timer); + struct rfcomm_session *s = timer_container_of(s, t, timer); BT_DBG("session %p state %ld", s, s->state); @@ -260,7 +260,7 @@ static void rfcomm_session_clear_timer(struct rfcomm_session *s) /* ---- RFCOMM DLCs ---- */ static void rfcomm_dlc_timeout(struct timer_list *t) { - struct rfcomm_dlc *d = from_timer(d, t, timer); + struct rfcomm_dlc *d = timer_container_of(d, t, timer); BT_DBG("dlc %p state %ld", d, d->state); diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c index fb6f7f2001c9..0224ef3dfec0 100644 --- a/net/bridge/br_multicast.c +++ b/net/bridge/br_multicast.c @@ -648,7 +648,7 @@ static void br_multicast_del_mdb_entry(struct net_bridge_mdb_entry *mp) static void br_multicast_group_expired(struct timer_list *t) { - struct net_bridge_mdb_entry *mp = from_timer(mp, t, timer); + struct net_bridge_mdb_entry *mp = timer_container_of(mp, t, timer); struct net_bridge *br = mp->br; spin_lock(&br->multicast_lock); @@ -856,7 +856,7 @@ static void br_multicast_find_del_pg(struct net_bridge *br, static void br_multicast_port_group_expired(struct timer_list *t) { - struct net_bridge_port_group *pg = from_timer(pg, t, timer); + struct net_bridge_port_group *pg = timer_container_of(pg, t, timer); struct net_bridge_group_src *src_ent; struct net_bridge *br = pg->key.port->br; struct hlist_node *tmp; @@ -1314,7 +1314,7 @@ struct net_bridge_mdb_entry *br_multicast_new_group(struct net_bridge *br, static void br_multicast_group_src_expired(struct timer_list *t) { - struct net_bridge_group_src *src = from_timer(src, t, timer); + struct net_bridge_group_src *src = timer_container_of(src, t, timer); struct net_bridge_port_group *pg; struct net_bridge *br = src->br; @@ -1667,8 +1667,8 @@ out: static void br_ip4_multicast_router_expired(struct timer_list *t) { - struct net_bridge_mcast_port *pmctx = from_timer(pmctx, t, - ip4_mc_router_timer); + struct net_bridge_mcast_port *pmctx = timer_container_of(pmctx, t, + ip4_mc_router_timer); br_multicast_router_expired(pmctx, t, &pmctx->ip4_rlist); } @@ -1676,8 +1676,8 @@ static void br_ip4_multicast_router_expired(struct timer_list *t) #if IS_ENABLED(CONFIG_IPV6) static void br_ip6_multicast_router_expired(struct timer_list *t) { - struct net_bridge_mcast_port *pmctx = from_timer(pmctx, t, - ip6_mc_router_timer); + struct net_bridge_mcast_port *pmctx = timer_container_of(pmctx, t, + ip6_mc_router_timer); br_multicast_router_expired(pmctx, t, &pmctx->ip6_rlist); } @@ -1713,8 +1713,8 @@ out: static void br_ip4_multicast_local_router_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip4_mc_router_timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip4_mc_router_timer); br_multicast_local_router_expired(brmctx, t); } @@ -1722,8 +1722,8 @@ static void br_ip4_multicast_local_router_expired(struct timer_list *t) #if IS_ENABLED(CONFIG_IPV6) static void br_ip6_multicast_local_router_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip6_mc_router_timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip6_mc_router_timer); br_multicast_local_router_expired(brmctx, t); } @@ -1746,8 +1746,8 @@ out: static void br_ip4_multicast_querier_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip4_other_query.timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip4_other_query.timer); br_multicast_querier_expired(brmctx, &brmctx->ip4_own_query); } @@ -1755,8 +1755,8 @@ static void br_ip4_multicast_querier_expired(struct timer_list *t) #if IS_ENABLED(CONFIG_IPV6) static void br_ip6_multicast_querier_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip6_other_query.timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip6_other_query.timer); br_multicast_querier_expired(brmctx, &brmctx->ip6_own_query); } @@ -1918,8 +1918,8 @@ out: static void br_ip4_multicast_port_query_expired(struct timer_list *t) { - struct net_bridge_mcast_port *pmctx = from_timer(pmctx, t, - ip4_own_query.timer); + struct net_bridge_mcast_port *pmctx = timer_container_of(pmctx, t, + ip4_own_query.timer); br_multicast_port_query_expired(pmctx, &pmctx->ip4_own_query); } @@ -1927,8 +1927,8 @@ static void br_ip4_multicast_port_query_expired(struct timer_list *t) #if IS_ENABLED(CONFIG_IPV6) static void br_ip6_multicast_port_query_expired(struct timer_list *t) { - struct net_bridge_mcast_port *pmctx = from_timer(pmctx, t, - ip6_own_query.timer); + struct net_bridge_mcast_port *pmctx = timer_container_of(pmctx, t, + ip6_own_query.timer); br_multicast_port_query_expired(pmctx, &pmctx->ip6_own_query); } @@ -1936,7 +1936,8 @@ static void br_ip6_multicast_port_query_expired(struct timer_list *t) static void br_multicast_port_group_rexmit(struct timer_list *t) { - struct net_bridge_port_group *pg = from_timer(pg, t, rexmit_timer); + struct net_bridge_port_group *pg = timer_container_of(pg, t, + rexmit_timer); struct bridge_mcast_other_query *other_query = NULL; struct net_bridge *br = pg->key.port->br; struct net_bridge_mcast_port *pmctx; @@ -4056,8 +4057,8 @@ out: static void br_ip4_multicast_query_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip4_own_query.timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip4_own_query.timer); br_multicast_query_expired(brmctx, &brmctx->ip4_own_query, &brmctx->ip4_querier); @@ -4066,8 +4067,8 @@ static void br_ip4_multicast_query_expired(struct timer_list *t) #if IS_ENABLED(CONFIG_IPV6) static void br_ip6_multicast_query_expired(struct timer_list *t) { - struct net_bridge_mcast *brmctx = from_timer(brmctx, t, - ip6_own_query.timer); + struct net_bridge_mcast *brmctx = timer_container_of(brmctx, t, + ip6_own_query.timer); br_multicast_query_expired(brmctx, &brmctx->ip6_own_query, &brmctx->ip6_querier); diff --git a/net/bridge/br_multicast_eht.c b/net/bridge/br_multicast_eht.c index c126aa4e7551..adfd74102019 100644 --- a/net/bridge/br_multicast_eht.c +++ b/net/bridge/br_multicast_eht.c @@ -207,7 +207,9 @@ void br_multicast_eht_clean_sets(struct net_bridge_port_group *pg) static void br_multicast_eht_set_entry_expired(struct timer_list *t) { - struct net_bridge_group_eht_set_entry *set_h = from_timer(set_h, t, timer); + struct net_bridge_group_eht_set_entry *set_h = timer_container_of(set_h, + t, + timer); struct net_bridge *br = set_h->br; spin_lock(&br->multicast_lock); @@ -223,8 +225,9 @@ out: static void br_multicast_eht_set_expired(struct timer_list *t) { - struct net_bridge_group_eht_set *eht_set = from_timer(eht_set, t, - timer); + struct net_bridge_group_eht_set *eht_set = timer_container_of(eht_set, + t, + timer); struct net_bridge *br = eht_set->br; spin_lock(&br->multicast_lock); diff --git a/net/bridge/br_stp_timer.c b/net/bridge/br_stp_timer.c index 27bf1979b909..e5d453305381 100644 --- a/net/bridge/br_stp_timer.c +++ b/net/bridge/br_stp_timer.c @@ -29,7 +29,7 @@ static int br_is_designated_for_some_port(const struct net_bridge *br) static void br_hello_timer_expired(struct timer_list *t) { - struct net_bridge *br = from_timer(br, t, hello_timer); + struct net_bridge *br = timer_container_of(br, t, hello_timer); br_debug(br, "hello timer expired\n"); spin_lock(&br->lock); @@ -45,7 +45,8 @@ static void br_hello_timer_expired(struct timer_list *t) static void br_message_age_timer_expired(struct timer_list *t) { - struct net_bridge_port *p = from_timer(p, t, message_age_timer); + struct net_bridge_port *p = timer_container_of(p, t, + message_age_timer); struct net_bridge *br = p->br; const bridge_id *id = &p->designated_bridge; int was_root; @@ -78,7 +79,8 @@ static void br_message_age_timer_expired(struct timer_list *t) static void br_forward_delay_timer_expired(struct timer_list *t) { - struct net_bridge_port *p = from_timer(p, t, forward_delay_timer); + struct net_bridge_port *p = timer_container_of(p, t, + forward_delay_timer); struct net_bridge *br = p->br; br_debug(br, "port %u(%s) forward delay timer\n", @@ -102,7 +104,7 @@ static void br_forward_delay_timer_expired(struct timer_list *t) static void br_tcn_timer_expired(struct timer_list *t) { - struct net_bridge *br = from_timer(br, t, tcn_timer); + struct net_bridge *br = timer_container_of(br, t, tcn_timer); br_debug(br, "tcn timer expired\n"); spin_lock(&br->lock); @@ -116,7 +118,8 @@ static void br_tcn_timer_expired(struct timer_list *t) static void br_topology_change_timer_expired(struct timer_list *t) { - struct net_bridge *br = from_timer(br, t, topology_change_timer); + struct net_bridge *br = timer_container_of(br, t, + topology_change_timer); br_debug(br, "topo change timer expired\n"); spin_lock(&br->lock); @@ -127,7 +130,7 @@ static void br_topology_change_timer_expired(struct timer_list *t) static void br_hold_timer_expired(struct timer_list *t) { - struct net_bridge_port *p = from_timer(p, t, hold_timer); + struct net_bridge_port *p = timer_container_of(p, t, hold_timer); br_debug(p->br, "port %u(%s) hold timer expired\n", (unsigned int) p->port_no, p->dev->name); diff --git a/net/can/proc.c b/net/can/proc.c index 25fdf060e30d..0938bf7dd646 100644 --- a/net/can/proc.c +++ b/net/can/proc.c @@ -114,7 +114,7 @@ static unsigned long calc_rate(unsigned long oldjif, unsigned long newjif, void can_stat_update(struct timer_list *t) { - struct net *net = from_timer(net, t, can.stattimer); + struct net *net = timer_container_of(net, t, can.stattimer); struct can_pkg_stats *pkg_stats = net->can.pkg_stats; unsigned long j = jiffies; /* snapshot */ diff --git a/net/core/drop_monitor.c b/net/core/drop_monitor.c index 8a7ce640f74d..60d31c2feed3 100644 --- a/net/core/drop_monitor.c +++ b/net/core/drop_monitor.c @@ -208,7 +208,7 @@ static void send_dm_alert(struct work_struct *work) */ static void sched_send_work(struct timer_list *t) { - struct per_cpu_dm_data *data = from_timer(data, t, send_timer); + struct per_cpu_dm_data *data = timer_container_of(data, t, send_timer); schedule_work(&data->dm_alert_work); } diff --git a/net/core/filter.c b/net/core/filter.c index 327ca73f9cd7..7a72f766aacf 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3233,6 +3233,13 @@ static const struct bpf_func_proto bpf_skb_vlan_pop_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +static void bpf_skb_change_protocol(struct sk_buff *skb, u16 proto) +{ + skb->protocol = htons(proto); + if (skb_valid_dst(skb)) + skb_dst_drop(skb); +} + static int bpf_skb_generic_push(struct sk_buff *skb, u32 off, u32 len) { /* Caller already did skb_cow() with len as headroom, @@ -3329,7 +3336,7 @@ static int bpf_skb_proto_4_to_6(struct sk_buff *skb) } } - skb->protocol = htons(ETH_P_IPV6); + bpf_skb_change_protocol(skb, ETH_P_IPV6); skb_clear_hash(skb); return 0; @@ -3359,7 +3366,7 @@ static int bpf_skb_proto_6_to_4(struct sk_buff *skb) } } - skb->protocol = htons(ETH_P_IP); + bpf_skb_change_protocol(skb, ETH_P_IP); skb_clear_hash(skb); return 0; @@ -3550,10 +3557,10 @@ static int bpf_skb_net_grow(struct sk_buff *skb, u32 off, u32 len_diff, /* Match skb->protocol to new outer l3 protocol */ if (skb->protocol == htons(ETH_P_IP) && flags & BPF_F_ADJ_ROOM_ENCAP_L3_IPV6) - skb->protocol = htons(ETH_P_IPV6); + bpf_skb_change_protocol(skb, ETH_P_IPV6); else if (skb->protocol == htons(ETH_P_IPV6) && flags & BPF_F_ADJ_ROOM_ENCAP_L3_IPV4) - skb->protocol = htons(ETH_P_IP); + bpf_skb_change_protocol(skb, ETH_P_IP); } if (skb_is_gso(skb)) { @@ -3606,10 +3613,10 @@ static int bpf_skb_net_shrink(struct sk_buff *skb, u32 off, u32 len_diff, /* Match skb->protocol to new outer l3 protocol */ if (skb->protocol == htons(ETH_P_IP) && flags & BPF_F_ADJ_ROOM_DECAP_L3_IPV6) - skb->protocol = htons(ETH_P_IPV6); + bpf_skb_change_protocol(skb, ETH_P_IPV6); else if (skb->protocol == htons(ETH_P_IPV6) && flags & BPF_F_ADJ_ROOM_DECAP_L3_IPV4) - skb->protocol = htons(ETH_P_IP); + bpf_skb_change_protocol(skb, ETH_P_IP); if (skb_is_gso(skb)) { struct skb_shared_info *shinfo = skb_shinfo(skb); diff --git a/net/core/gen_estimator.c b/net/core/gen_estimator.c index 2b821b9a8699..7d426a8e29f3 100644 --- a/net/core/gen_estimator.c +++ b/net/core/gen_estimator.c @@ -75,7 +75,7 @@ static void est_fetch_counters(struct net_rate_estimator *e, static void est_timer(struct timer_list *t) { - struct net_rate_estimator *est = from_timer(est, t, timer); + struct net_rate_estimator *est = timer_container_of(est, t, timer); struct gnet_stats_basic_sync b; u64 b_bytes, b_packets; u64 rate, brate; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index a6e2c91ec3e7..49dce9a82295 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -1031,7 +1031,7 @@ static void neigh_probe(struct neighbour *neigh) static void neigh_timer_handler(struct timer_list *t) { unsigned long now, next; - struct neighbour *neigh = from_timer(neigh, t, timer); + struct neighbour *neigh = timer_container_of(neigh, t, timer); unsigned int state; int notify = 0; @@ -1569,7 +1569,7 @@ static void neigh_managed_work(struct work_struct *work) static void neigh_proxy_process(struct timer_list *t) { - struct neigh_table *tbl = from_timer(tbl, t, proxy_timer); + struct neigh_table *tbl = timer_container_of(tbl, t, proxy_timer); long sched_next = 0; unsigned long now = jiffies; struct sk_buff *skb, *n; diff --git a/net/ethtool/ioctl.c b/net/ethtool/ioctl.c index 39ec920f5de7..71c828d0bf31 100644 --- a/net/ethtool/ioctl.c +++ b/net/ethtool/ioctl.c @@ -1083,7 +1083,8 @@ static noinline_for_stack int ethtool_set_rxnfc(struct net_device *dev, ethtool_get_flow_spec_ring(info.fs.ring_cookie)) return -EINVAL; - if (!xa_load(&dev->ethtool->rss_ctx, info.rss_context)) + if (info.rss_context && + !xa_load(&dev->ethtool->rss_ctx, info.rss_context)) return -EINVAL; } diff --git a/net/ethtool/mm.c b/net/ethtool/mm.c index ad9b40034003..29bbbc149375 100644 --- a/net/ethtool/mm.c +++ b/net/ethtool/mm.c @@ -315,7 +315,7 @@ static void ethtool_mmsv_send_mpacket(struct ethtool_mmsv *mmsv, */ static void ethtool_mmsv_verify_timer(struct timer_list *t) { - struct ethtool_mmsv *mmsv = from_timer(mmsv, t, verify_timer); + struct ethtool_mmsv *mmsv = timer_container_of(mmsv, t, verify_timer); unsigned long flags; bool rearm = false; diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c index 0d1e56965af0..88657255fec1 100644 --- a/net/hsr/hsr_device.c +++ b/net/hsr/hsr_device.c @@ -408,7 +408,7 @@ static void hsr_announce(struct timer_list *t) struct hsr_port *master; unsigned long interval; - hsr = from_timer(hsr, t, announce_timer); + hsr = timer_container_of(hsr, t, announce_timer); rcu_read_lock(); master = hsr_port_get_hsr(hsr, HSR_PT_MASTER); @@ -424,7 +424,8 @@ static void hsr_announce(struct timer_list *t) */ static void hsr_proxy_announce(struct timer_list *t) { - struct hsr_priv *hsr = from_timer(hsr, t, announce_proxy_timer); + struct hsr_priv *hsr = timer_container_of(hsr, t, + announce_proxy_timer); struct hsr_port *interlink; unsigned long interval = 0; struct hsr_node *node; diff --git a/net/hsr/hsr_framereg.c b/net/hsr/hsr_framereg.c index 4ce471a2f387..3a2a2fa7a0a3 100644 --- a/net/hsr/hsr_framereg.c +++ b/net/hsr/hsr_framereg.c @@ -617,7 +617,7 @@ static struct hsr_port *get_late_port(struct hsr_priv *hsr, */ void hsr_prune_nodes(struct timer_list *t) { - struct hsr_priv *hsr = from_timer(hsr, t, prune_timer); + struct hsr_priv *hsr = timer_container_of(hsr, t, prune_timer); struct hsr_node *node; struct hsr_node *tmp; struct hsr_port *port; @@ -685,7 +685,7 @@ void hsr_prune_nodes(struct timer_list *t) void hsr_prune_proxy_nodes(struct timer_list *t) { - struct hsr_priv *hsr = from_timer(hsr, t, prune_proxy_timer); + struct hsr_priv *hsr = timer_container_of(hsr, t, prune_proxy_timer); unsigned long timestamp; struct hsr_node *node; struct hsr_node *tmp; diff --git a/net/ieee802154/6lowpan/reassembly.c b/net/ieee802154/6lowpan/reassembly.c index d4b983d17038..ddb6a5817d09 100644 --- a/net/ieee802154/6lowpan/reassembly.c +++ b/net/ieee802154/6lowpan/reassembly.c @@ -44,7 +44,7 @@ static void lowpan_frag_init(struct inet_frag_queue *q, const void *a) static void lowpan_frag_expire(struct timer_list *t) { - struct inet_frag_queue *frag = from_timer(frag, t, timer); + struct inet_frag_queue *frag = timer_container_of(frag, t, timer); struct frag_queue *fq; int refs = 1; diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c index ca7d539b3846..d1769034b643 100644 --- a/net/ipv4/igmp.c +++ b/net/ipv4/igmp.c @@ -801,7 +801,7 @@ static int igmp_send_report(struct in_device *in_dev, struct ip_mc_list *pmc, static void igmp_gq_timer_expire(struct timer_list *t) { - struct in_device *in_dev = from_timer(in_dev, t, mr_gq_timer); + struct in_device *in_dev = timer_container_of(in_dev, t, mr_gq_timer); in_dev->mr_gq_running = 0; igmpv3_send_report(in_dev, NULL); @@ -810,7 +810,7 @@ static void igmp_gq_timer_expire(struct timer_list *t) static void igmp_ifc_timer_expire(struct timer_list *t) { - struct in_device *in_dev = from_timer(in_dev, t, mr_ifc_timer); + struct in_device *in_dev = timer_container_of(in_dev, t, mr_ifc_timer); u32 mr_ifc_count; igmpv3_send_cr(in_dev); @@ -840,7 +840,7 @@ static void igmp_ifc_event(struct in_device *in_dev) static void igmp_timer_expire(struct timer_list *t) { - struct ip_mc_list *im = from_timer(im, t, timer); + struct ip_mc_list *im = timer_container_of(im, t, timer); struct in_device *in_dev = im->interface; spin_lock(&im->lock); diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 20915895bdaa..6906bedad19a 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -1065,7 +1065,7 @@ EXPORT_IPV6_MOD(inet_csk_reqsk_queue_drop_and_put); static void reqsk_timer_handler(struct timer_list *t) { - struct request_sock *req = from_timer(req, t, rsk_timer); + struct request_sock *req = timer_container_of(req, t, rsk_timer); struct request_sock *nreq = NULL, *oreq = req; struct sock *sk_listener = req->rsk_listener; struct inet_connection_sock *icsk; diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c index 67efe9501581..875ff923a8ed 100644 --- a/net/ipv4/inet_timewait_sock.c +++ b/net/ipv4/inet_timewait_sock.c @@ -169,7 +169,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw, static void tw_timer_handler(struct timer_list *t) { - struct inet_timewait_sock *tw = from_timer(tw, t, tw_timer); + struct inet_timewait_sock *tw = timer_container_of(tw, t, tw_timer); inet_twsk_kill(tw); } diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c index 77f395b28ec7..64b3fb3208af 100644 --- a/net/ipv4/ip_fragment.c +++ b/net/ipv4/ip_fragment.c @@ -123,7 +123,7 @@ static bool frag_expire_skip_icmp(u32 user) static void ip_expire(struct timer_list *t) { enum skb_drop_reason reason = SKB_DROP_REASON_FRAG_REASM_TIMEOUT; - struct inet_frag_queue *frag = from_timer(frag, t, timer); + struct inet_frag_queue *frag = timer_container_of(frag, t, timer); const struct iphdr *iph; struct sk_buff *head = NULL; struct net *net; diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c index 2ff2f79c7351..a7d09ae9d761 100644 --- a/net/ipv4/ipmr.c +++ b/net/ipv4/ipmr.c @@ -765,7 +765,7 @@ static void ipmr_destroy_unres(struct mr_table *mrt, struct mfc_cache *c) /* Timer process for the unresolved queue. */ static void ipmr_expire_process(struct timer_list *t) { - struct mr_table *mrt = from_timer(mrt, t, ipmr_expire_timer); + struct mr_table *mrt = timer_container_of(mrt, t, ipmr_expire_timer); struct mr_mfc *c, *next; unsigned long expires; unsigned long now; diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c index e4c616bbd727..bb37e24b97a7 100644 --- a/net/ipv4/tcp_timer.c +++ b/net/ipv4/tcp_timer.c @@ -359,7 +359,7 @@ void tcp_delack_timer_handler(struct sock *sk) static void tcp_delack_timer(struct timer_list *t) { struct inet_connection_sock *icsk = - from_timer(icsk, t, icsk_delack_timer); + timer_container_of(icsk, t, icsk_delack_timer); struct sock *sk = &icsk->icsk_inet.sk; /* Avoid taking socket spinlock if there is no ACK to send. @@ -726,7 +726,7 @@ void tcp_write_timer_handler(struct sock *sk) static void tcp_write_timer(struct timer_list *t) { struct inet_connection_sock *icsk = - from_timer(icsk, t, icsk_retransmit_timer); + timer_container_of(icsk, t, icsk_retransmit_timer); struct sock *sk = &icsk->icsk_inet.sk; /* Avoid locking the socket when there is no pending event. */ @@ -778,7 +778,7 @@ EXPORT_IPV6_MOD_GPL(tcp_set_keepalive); static void tcp_keepalive_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); u32 elapsed; diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c index 43b19adfbf88..ba2ec7c870cc 100644 --- a/net/ipv6/addrconf.c +++ b/net/ipv6/addrconf.c @@ -4017,7 +4017,7 @@ restart: static void addrconf_rs_timer(struct timer_list *t) { - struct inet6_dev *idev = from_timer(idev, t, rs_timer); + struct inet6_dev *idev = timer_container_of(idev, t, rs_timer); struct net_device *dev = idev->dev; struct in6_addr lladdr; int rtr_solicits; diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c index 7094d7708686..93578b2ec35f 100644 --- a/net/ipv6/ip6_fib.c +++ b/net/ipv6/ip6_fib.c @@ -2442,7 +2442,7 @@ void fib6_run_gc(unsigned long expires, struct net *net, bool force) static void fib6_gc_timer_cb(struct timer_list *t) { - struct net *arg = from_timer(arg, t, ipv6.ip6_fib_timer); + struct net *arg = timer_container_of(arg, t, ipv6.ip6_fib_timer); fib6_run_gc(0, arg, true); } diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c index 3276cde5ebd7..9db31e5b998c 100644 --- a/net/ipv6/ip6mr.c +++ b/net/ipv6/ip6mr.c @@ -839,7 +839,7 @@ static void ipmr_do_expire_process(struct mr_table *mrt) static void ipmr_expire_process(struct timer_list *t) { - struct mr_table *mrt = from_timer(mrt, t, ipmr_expire_timer); + struct mr_table *mrt = timer_container_of(mrt, t, ipmr_expire_timer); if (!spin_trylock(&mfc_unres_lock)) { mod_timer(&mrt->ipmr_expire_timer, jiffies + 1); diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c index d6bd8f7079bb..64ab23ff559b 100644 --- a/net/ipv6/netfilter/nf_conntrack_reasm.c +++ b/net/ipv6/netfilter/nf_conntrack_reasm.c @@ -133,7 +133,7 @@ static inline u8 ip6_frag_ecn(const struct ipv6hdr *ipv6h) static void nf_ct_frag6_expire(struct timer_list *t) { - struct inet_frag_queue *frag = from_timer(frag, t, timer); + struct inet_frag_queue *frag = timer_container_of(frag, t, timer); struct frag_queue *fq; fq = container_of(frag, struct frag_queue, q); diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c index 49740898bc13..7d4bcf3fda5b 100644 --- a/net/ipv6/reassembly.c +++ b/net/ipv6/reassembly.c @@ -73,7 +73,7 @@ static int ip6_frag_reasm(struct frag_queue *fq, struct sk_buff *skb, static void ip6_frag_expire(struct timer_list *t) { - struct inet_frag_queue *frag = from_timer(frag, t, timer); + struct inet_frag_queue *frag = timer_container_of(frag, t, timer); struct frag_queue *fq; fq = container_of(frag, struct frag_queue, q); diff --git a/net/ipv6/route.c b/net/ipv6/route.c index 0143262094b0..79c8f1acf8a3 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -3737,6 +3737,53 @@ void fib6_nh_release_dsts(struct fib6_nh *fib6_nh) } } +static int fib6_config_validate(struct fib6_config *cfg, + struct netlink_ext_ack *extack) +{ + /* RTF_PCPU is an internal flag; can not be set by userspace */ + if (cfg->fc_flags & RTF_PCPU) { + NL_SET_ERR_MSG(extack, "Userspace can not set RTF_PCPU"); + goto errout; + } + + /* RTF_CACHE is an internal flag; can not be set by userspace */ + if (cfg->fc_flags & RTF_CACHE) { + NL_SET_ERR_MSG(extack, "Userspace can not set RTF_CACHE"); + goto errout; + } + + if (cfg->fc_type > RTN_MAX) { + NL_SET_ERR_MSG(extack, "Invalid route type"); + goto errout; + } + + if (cfg->fc_dst_len > 128) { + NL_SET_ERR_MSG(extack, "Invalid prefix length"); + goto errout; + } + +#ifdef CONFIG_IPV6_SUBTREES + if (cfg->fc_src_len > 128) { + NL_SET_ERR_MSG(extack, "Invalid source address length"); + goto errout; + } + + if (cfg->fc_nh_id && cfg->fc_src_len) { + NL_SET_ERR_MSG(extack, "Nexthops can not be used with source routing"); + goto errout; + } +#else + if (cfg->fc_src_len) { + NL_SET_ERR_MSG(extack, + "Specifying source address requires IPV6_SUBTREES to be enabled"); + goto errout; + } +#endif + return 0; +errout: + return -EINVAL; +} + static struct fib6_info *ip6_route_info_create(struct fib6_config *cfg, gfp_t gfp_flags, struct netlink_ext_ack *extack) @@ -3886,6 +3933,10 @@ int ip6_route_add(struct fib6_config *cfg, gfp_t gfp_flags, struct fib6_info *rt; int err; + err = fib6_config_validate(cfg, extack); + if (err) + return err; + rt = ip6_route_info_create(cfg, gfp_flags, extack); if (IS_ERR(rt)) return PTR_ERR(rt); @@ -4479,53 +4530,6 @@ void rt6_purge_dflt_routers(struct net *net) rcu_read_unlock(); } -static int fib6_config_validate(struct fib6_config *cfg, - struct netlink_ext_ack *extack) -{ - /* RTF_PCPU is an internal flag; can not be set by userspace */ - if (cfg->fc_flags & RTF_PCPU) { - NL_SET_ERR_MSG(extack, "Userspace can not set RTF_PCPU"); - goto errout; - } - - /* RTF_CACHE is an internal flag; can not be set by userspace */ - if (cfg->fc_flags & RTF_CACHE) { - NL_SET_ERR_MSG(extack, "Userspace can not set RTF_CACHE"); - goto errout; - } - - if (cfg->fc_type > RTN_MAX) { - NL_SET_ERR_MSG(extack, "Invalid route type"); - goto errout; - } - - if (cfg->fc_dst_len > 128) { - NL_SET_ERR_MSG(extack, "Invalid prefix length"); - goto errout; - } - -#ifdef CONFIG_IPV6_SUBTREES - if (cfg->fc_src_len > 128) { - NL_SET_ERR_MSG(extack, "Invalid source address length"); - goto errout; - } - - if (cfg->fc_nh_id && cfg->fc_src_len) { - NL_SET_ERR_MSG(extack, "Nexthops can not be used with source routing"); - goto errout; - } -#else - if (cfg->fc_src_len) { - NL_SET_ERR_MSG(extack, - "Specifying source address requires IPV6_SUBTREES to be enabled"); - goto errout; - } -#endif - return 0; -errout: - return -EINVAL; -} - static void rtmsg_to_fib6_config(struct net *net, struct in6_rtmsg *rtmsg, struct fib6_config *cfg) @@ -4563,10 +4567,6 @@ int ipv6_route_ioctl(struct net *net, unsigned int cmd, struct in6_rtmsg *rtmsg) switch (cmd) { case SIOCADDRT: - err = fib6_config_validate(&cfg, NULL); - if (err) - break; - /* Only do the default setting of fc_metric in route adding */ if (cfg.fc_metric == 0) cfg.fc_metric = IP6_RT_PRIO_USER; @@ -5402,6 +5402,10 @@ static int ip6_route_multipath_add(struct fib6_config *cfg, int nhn = 0; int err; + err = fib6_config_validate(cfg, extack); + if (err) + return err; + replace = (cfg->fc_nlinfo.nlh && (cfg->fc_nlinfo.nlh->nlmsg_flags & NLM_F_REPLACE)); @@ -5636,10 +5640,6 @@ static int inet6_rtm_newroute(struct sk_buff *skb, struct nlmsghdr *nlh, if (err < 0) return err; - err = fib6_config_validate(&cfg, extack); - if (err) - return err; - if (cfg.fc_metric == 0) cfg.fc_metric = IP6_RT_PRIO_USER; diff --git a/net/lapb/lapb_timer.c b/net/lapb/lapb_timer.c index 5b3f3b444d19..9fde6cf20f10 100644 --- a/net/lapb/lapb_timer.c +++ b/net/lapb/lapb_timer.c @@ -74,7 +74,7 @@ int lapb_t1timer_running(struct lapb_cb *lapb) static void lapb_t2timer_expiry(struct timer_list *t) { - struct lapb_cb *lapb = from_timer(lapb, t, t2timer); + struct lapb_cb *lapb = timer_container_of(lapb, t, t2timer); spin_lock_bh(&lapb->lock); if (timer_pending(&lapb->t2timer)) /* A new timer has been set up */ @@ -94,7 +94,7 @@ out: static void lapb_t1timer_expiry(struct timer_list *t) { - struct lapb_cb *lapb = from_timer(lapb, t, t1timer); + struct lapb_cb *lapb = timer_container_of(lapb, t, t1timer); spin_lock_bh(&lapb->lock); if (timer_pending(&lapb->t1timer)) /* A new timer has been set up */ diff --git a/net/llc/llc_c_ac.c b/net/llc/llc_c_ac.c index 7e8fc710c590..0779daa8aa8f 100644 --- a/net/llc/llc_c_ac.c +++ b/net/llc/llc_c_ac.c @@ -1335,28 +1335,31 @@ static void llc_conn_tmr_common_cb(struct sock *sk, u8 type) void llc_conn_pf_cycle_tmr_cb(struct timer_list *t) { - struct llc_sock *llc = from_timer(llc, t, pf_cycle_timer.timer); + struct llc_sock *llc = timer_container_of(llc, t, + pf_cycle_timer.timer); llc_conn_tmr_common_cb(&llc->sk, LLC_CONN_EV_TYPE_P_TMR); } void llc_conn_busy_tmr_cb(struct timer_list *t) { - struct llc_sock *llc = from_timer(llc, t, busy_state_timer.timer); + struct llc_sock *llc = timer_container_of(llc, t, + busy_state_timer.timer); llc_conn_tmr_common_cb(&llc->sk, LLC_CONN_EV_TYPE_BUSY_TMR); } void llc_conn_ack_tmr_cb(struct timer_list *t) { - struct llc_sock *llc = from_timer(llc, t, ack_timer.timer); + struct llc_sock *llc = timer_container_of(llc, t, ack_timer.timer); llc_conn_tmr_common_cb(&llc->sk, LLC_CONN_EV_TYPE_ACK_TMR); } void llc_conn_rej_tmr_cb(struct timer_list *t) { - struct llc_sock *llc = from_timer(llc, t, rej_sent_timer.timer); + struct llc_sock *llc = timer_container_of(llc, t, + rej_sent_timer.timer); llc_conn_tmr_common_cb(&llc->sk, LLC_CONN_EV_TYPE_REJ_TMR); } diff --git a/net/mac80211/agg-rx.c b/net/mac80211/agg-rx.c index 85612234742a..ee534797c033 100644 --- a/net/mac80211/agg-rx.c +++ b/net/mac80211/agg-rx.c @@ -143,7 +143,8 @@ EXPORT_SYMBOL(ieee80211_stop_rx_ba_session); */ static void sta_rx_agg_session_timer_expired(struct timer_list *t) { - struct tid_ampdu_rx *tid_rx = from_timer(tid_rx, t, session_timer); + struct tid_ampdu_rx *tid_rx = timer_container_of(tid_rx, t, + session_timer); struct sta_info *sta = tid_rx->sta; u8 tid = tid_rx->tid; unsigned long timeout; @@ -163,7 +164,8 @@ static void sta_rx_agg_session_timer_expired(struct timer_list *t) static void sta_rx_agg_reorder_timer_expired(struct timer_list *t) { - struct tid_ampdu_rx *tid_rx = from_timer(tid_rx, t, reorder_timer); + struct tid_ampdu_rx *tid_rx = timer_container_of(tid_rx, t, + reorder_timer); rcu_read_lock(); ieee80211_release_reorder_timeout(tid_rx->sta, tid_rx->tid); diff --git a/net/mac80211/agg-tx.c b/net/mac80211/agg-tx.c index 8dc8c3c96b96..dbd9ad5f3992 100644 --- a/net/mac80211/agg-tx.c +++ b/net/mac80211/agg-tx.c @@ -422,7 +422,8 @@ int __ieee80211_stop_tx_ba_session(struct sta_info *sta, u16 tid, */ static void sta_addba_resp_timer_expired(struct timer_list *t) { - struct tid_ampdu_tx *tid_tx = from_timer(tid_tx, t, addba_resp_timer); + struct tid_ampdu_tx *tid_tx = timer_container_of(tid_tx, t, + addba_resp_timer); struct sta_info *sta = tid_tx->sta; u8 tid = tid_tx->tid; @@ -574,7 +575,8 @@ EXPORT_SYMBOL(ieee80211_refresh_tx_agg_session_timer); */ static void sta_tx_agg_session_timer_expired(struct timer_list *t) { - struct tid_ampdu_tx *tid_tx = from_timer(tid_tx, t, session_timer); + struct tid_ampdu_tx *tid_tx = timer_container_of(tid_tx, t, + session_timer); struct sta_info *sta = tid_tx->sta; u8 tid = tid_tx->tid; unsigned long timeout; diff --git a/net/mac80211/ibss.c b/net/mac80211/ibss.c index a6e7b7ba6a01..9ed87d6f5019 100644 --- a/net/mac80211/ibss.c +++ b/net/mac80211/ibss.c @@ -1673,7 +1673,7 @@ void ieee80211_ibss_work(struct ieee80211_sub_if_data *sdata) static void ieee80211_ibss_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.ibss.timer); + timer_container_of(sdata, t, u.ibss.timer); wiphy_work_queue(sdata->local->hw.wiphy, &sdata->work); } diff --git a/net/mac80211/led.c b/net/mac80211/led.c index 885fa6aa3fc1..fabbffdd3ac2 100644 --- a/net/mac80211/led.c +++ b/net/mac80211/led.c @@ -257,7 +257,8 @@ static unsigned long tpt_trig_traffic(struct ieee80211_local *local, static void tpt_trig_timer(struct timer_list *t) { - struct tpt_led_trigger *tpt_trig = from_timer(tpt_trig, t, timer); + struct tpt_led_trigger *tpt_trig = timer_container_of(tpt_trig, t, + timer); struct ieee80211_local *local = tpt_trig->local; unsigned long on, off, tpt; int i; diff --git a/net/mac80211/mesh.c b/net/mac80211/mesh.c index 5cc56d578048..d00d9d413c5c 100644 --- a/net/mac80211/mesh.c +++ b/net/mac80211/mesh.c @@ -40,7 +40,7 @@ void ieee80211s_stop(void) static void ieee80211_mesh_housekeeping_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mesh.housekeeping_timer); + timer_container_of(sdata, t, u.mesh.housekeeping_timer); struct ieee80211_local *local = sdata->local; struct ieee80211_if_mesh *ifmsh = &sdata->u.mesh; @@ -684,7 +684,7 @@ int mesh_add_eht_oper_ie(struct ieee80211_sub_if_data *sdata, struct sk_buff *sk static void ieee80211_mesh_path_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mesh.mesh_path_timer); + timer_container_of(sdata, t, u.mesh.mesh_path_timer); wiphy_work_queue(sdata->local->hw.wiphy, &sdata->work); } @@ -692,7 +692,7 @@ static void ieee80211_mesh_path_timer(struct timer_list *t) static void ieee80211_mesh_path_root_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mesh.mesh_path_root_timer); + timer_container_of(sdata, t, u.mesh.mesh_path_root_timer); struct ieee80211_if_mesh *ifmsh = &sdata->u.mesh; set_bit(MESH_WORK_ROOT, &ifmsh->wrkq_flags); diff --git a/net/mac80211/mesh_hwmp.c b/net/mac80211/mesh_hwmp.c index 91444301a84a..9101858525dd 100644 --- a/net/mac80211/mesh_hwmp.c +++ b/net/mac80211/mesh_hwmp.c @@ -1292,7 +1292,7 @@ int mesh_nexthop_lookup(struct ieee80211_sub_if_data *sdata, void mesh_path_timer(struct timer_list *t) { - struct mesh_path *mpath = from_timer(mpath, t, timer); + struct mesh_path *mpath = timer_container_of(mpath, t, timer); struct ieee80211_sub_if_data *sdata = mpath->sdata; int ret; diff --git a/net/mac80211/mesh_plink.c b/net/mac80211/mesh_plink.c index 9c6a2b342170..cb45a5d2009d 100644 --- a/net/mac80211/mesh_plink.c +++ b/net/mac80211/mesh_plink.c @@ -653,7 +653,7 @@ out: void mesh_plink_timer(struct timer_list *t) { - struct mesh_sta *mesh = from_timer(mesh, t, plink_timer); + struct mesh_sta *mesh = timer_container_of(mesh, t, plink_timer); struct sta_info *sta; u16 reason = 0; struct ieee80211_sub_if_data *sdata; diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index 948909a242d6..2d46d4af60d7 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -3383,7 +3383,8 @@ void ieee80211_dynamic_ps_enable_work(struct wiphy *wiphy, void ieee80211_dynamic_ps_timer(struct timer_list *t) { - struct ieee80211_local *local = from_timer(local, t, dynamic_ps_timer); + struct ieee80211_local *local = timer_container_of(local, t, + dynamic_ps_timer); wiphy_work_queue(local->hw.wiphy, &local->dynamic_ps_enable_work); } @@ -8079,7 +8080,7 @@ void ieee80211_sta_rx_queued_mgmt(struct ieee80211_sub_if_data *sdata, static void ieee80211_sta_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mgd.timer); + timer_container_of(sdata, t, u.mgd.timer); wiphy_work_queue(sdata->local->hw.wiphy, &sdata->work); } @@ -8385,7 +8386,7 @@ void ieee80211_sta_work(struct ieee80211_sub_if_data *sdata) static void ieee80211_sta_bcn_mon_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mgd.bcn_mon_timer); + timer_container_of(sdata, t, u.mgd.bcn_mon_timer); if (WARN_ON(ieee80211_vif_is_mld(&sdata->vif))) return; @@ -8405,7 +8406,7 @@ static void ieee80211_sta_bcn_mon_timer(struct timer_list *t) static void ieee80211_sta_conn_mon_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.mgd.conn_mon_timer); + timer_container_of(sdata, t, u.mgd.conn_mon_timer); struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; struct ieee80211_local *local = sdata->local; struct sta_info *sta; diff --git a/net/mac80211/ocb.c b/net/mac80211/ocb.c index ece1e83c7b2f..a5d4358f122a 100644 --- a/net/mac80211/ocb.c +++ b/net/mac80211/ocb.c @@ -146,7 +146,7 @@ void ieee80211_ocb_work(struct ieee80211_sub_if_data *sdata) static void ieee80211_ocb_housekeeping_timer(struct timer_list *t) { struct ieee80211_sub_if_data *sdata = - from_timer(sdata, t, u.ocb.housekeeping_timer); + timer_container_of(sdata, t, u.ocb.housekeeping_timer); struct ieee80211_local *local = sdata->local; struct ieee80211_if_ocb *ifocb = &sdata->u.ocb; diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index 84b18be1f0b1..61583173629e 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -1542,7 +1542,8 @@ int sta_info_destroy_addr_bss(struct ieee80211_sub_if_data *sdata, static void sta_info_cleanup(struct timer_list *t) { - struct ieee80211_local *local = from_timer(local, t, sta_cleanup); + struct ieee80211_local *local = timer_container_of(local, t, + sta_cleanup); struct sta_info *sta; bool timer_needed = false; diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c index 1306d4dc287b..feb01747d7d8 100644 --- a/net/mptcp/pm.c +++ b/net/mptcp/pm.c @@ -270,7 +270,8 @@ int mptcp_pm_mp_prio_send_ack(struct mptcp_sock *msk, static void mptcp_pm_add_timer(struct timer_list *timer) { - struct mptcp_pm_add_entry *entry = from_timer(entry, timer, add_timer); + struct mptcp_pm_add_entry *entry = timer_container_of(entry, timer, + add_timer); struct mptcp_sock *msk = entry->sock; struct sock *sk = (struct sock *)msk; diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 0749733ea897..edf14c2c2062 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -2203,8 +2203,8 @@ out_err: static void mptcp_retransmit_timer(struct timer_list *t) { - struct inet_connection_sock *icsk = from_timer(icsk, t, - icsk_retransmit_timer); + struct inet_connection_sock *icsk = timer_container_of(icsk, t, + icsk_retransmit_timer); struct sock *sk = &icsk->icsk_inet.sk; struct mptcp_sock *msk = mptcp_sk(sk); @@ -2223,7 +2223,7 @@ static void mptcp_retransmit_timer(struct timer_list *t) static void mptcp_tout_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); mptcp_schedule_work(sk); sock_put(sk); diff --git a/net/ncsi/ncsi-manage.c b/net/ncsi/ncsi-manage.c index b36947063783..446e4e3b9553 100644 --- a/net/ncsi/ncsi-manage.c +++ b/net/ncsi/ncsi-manage.c @@ -88,7 +88,7 @@ report: static void ncsi_channel_monitor(struct timer_list *t) { - struct ncsi_channel *nc = from_timer(nc, t, monitor.timer); + struct ncsi_channel *nc = timer_container_of(nc, t, monitor.timer); struct ncsi_package *np = nc->package; struct ncsi_dev_priv *ndp = np->ndp; struct ncsi_channel_mode *ncm; @@ -430,7 +430,7 @@ struct ncsi_dev *ncsi_find_dev(struct net_device *dev) static void ncsi_request_timeout(struct timer_list *t) { - struct ncsi_request *nr = from_timer(nr, t, timer); + struct ncsi_request *nr = timer_container_of(nr, t, timer); struct ncsi_dev_priv *ndp = nr->ndp; struct ncsi_cmd_pkt *cmd; struct ncsi_package *np; diff --git a/net/netfilter/ipset/ip_set_bitmap_gen.h b/net/netfilter/ipset/ip_set_bitmap_gen.h index 6ae042f702d2..798c7993635e 100644 --- a/net/netfilter/ipset/ip_set_bitmap_gen.h +++ b/net/netfilter/ipset/ip_set_bitmap_gen.h @@ -264,7 +264,7 @@ out: static void mtype_gc(struct timer_list *t) { - struct mtype *map = from_timer(map, t, gc); + struct mtype *map = timer_container_of(map, t, gc); struct ip_set *set = map->set; void *x; u32 id; diff --git a/net/netfilter/ipset/ip_set_list_set.c b/net/netfilter/ipset/ip_set_list_set.c index db794fe1300e..13c7a08aa868 100644 --- a/net/netfilter/ipset/ip_set_list_set.c +++ b/net/netfilter/ipset/ip_set_list_set.c @@ -571,7 +571,7 @@ static const struct ip_set_type_variant set_variant = { static void list_set_gc(struct timer_list *t) { - struct list_set *map = from_timer(map, t, gc); + struct list_set *map = timer_container_of(map, t, gc); struct ip_set *set = map->set; spin_lock_bh(&set->lock); diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c index 8699944c0baf..44b2ad695c15 100644 --- a/net/netfilter/ipvs/ip_vs_conn.c +++ b/net/netfilter/ipvs/ip_vs_conn.c @@ -846,7 +846,7 @@ static void ip_vs_conn_del_put(struct ip_vs_conn *cp) static void ip_vs_conn_expire(struct timer_list *t) { - struct ip_vs_conn *cp = from_timer(cp, t, timer); + struct ip_vs_conn *cp = timer_container_of(cp, t, timer); struct netns_ipvs *ipvs = cp->ipvs; /* diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c index 7d5b7418f8c7..6a6fc4478533 100644 --- a/net/netfilter/ipvs/ip_vs_ctl.c +++ b/net/netfilter/ipvs/ip_vs_ctl.c @@ -1331,7 +1331,8 @@ ip_vs_del_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest) static void ip_vs_dest_trash_expire(struct timer_list *t) { - struct netns_ipvs *ipvs = from_timer(ipvs, t, dest_trash_timer); + struct netns_ipvs *ipvs = timer_container_of(ipvs, t, + dest_trash_timer); struct ip_vs_dest *dest, *next; unsigned long now = jiffies; diff --git a/net/netfilter/ipvs/ip_vs_lblc.c b/net/netfilter/ipvs/ip_vs_lblc.c index 2423513d701d..156181a3bacd 100644 --- a/net/netfilter/ipvs/ip_vs_lblc.c +++ b/net/netfilter/ipvs/ip_vs_lblc.c @@ -292,7 +292,8 @@ static inline void ip_vs_lblc_full_check(struct ip_vs_service *svc) */ static void ip_vs_lblc_check_expire(struct timer_list *t) { - struct ip_vs_lblc_table *tbl = from_timer(tbl, t, periodic_timer); + struct ip_vs_lblc_table *tbl = timer_container_of(tbl, t, + periodic_timer); struct ip_vs_service *svc = tbl->svc; unsigned long now = jiffies; int goal; diff --git a/net/netfilter/ipvs/ip_vs_lblcr.c b/net/netfilter/ipvs/ip_vs_lblcr.c index cdb1d4bf6761..a021e6aba3d7 100644 --- a/net/netfilter/ipvs/ip_vs_lblcr.c +++ b/net/netfilter/ipvs/ip_vs_lblcr.c @@ -456,7 +456,8 @@ static inline void ip_vs_lblcr_full_check(struct ip_vs_service *svc) */ static void ip_vs_lblcr_check_expire(struct timer_list *t) { - struct ip_vs_lblcr_table *tbl = from_timer(tbl, t, periodic_timer); + struct ip_vs_lblcr_table *tbl = timer_container_of(tbl, t, + periodic_timer); struct ip_vs_service *svc = tbl->svc; unsigned long now = jiffies; int goal; diff --git a/net/netfilter/nf_conntrack_expect.c b/net/netfilter/nf_conntrack_expect.c index 21d22fa22e4e..cfc2daa3fc7f 100644 --- a/net/netfilter/nf_conntrack_expect.c +++ b/net/netfilter/nf_conntrack_expect.c @@ -71,7 +71,7 @@ EXPORT_SYMBOL_GPL(nf_ct_unlink_expect_report); static void nf_ct_expectation_timed_out(struct timer_list *t) { - struct nf_conntrack_expect *exp = from_timer(exp, t, timeout); + struct nf_conntrack_expect *exp = timer_container_of(exp, t, timeout); spin_lock_bh(&nf_conntrack_expect_lock); nf_ct_unlink_expect(exp); diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c index 882962f3c84d..bfcb9cd335bf 100644 --- a/net/netfilter/nfnetlink_log.c +++ b/net/netfilter/nfnetlink_log.c @@ -390,7 +390,7 @@ __nfulnl_flush(struct nfulnl_instance *inst) static void nfulnl_timer(struct timer_list *t) { - struct nfulnl_instance *inst = from_timer(inst, t, timer); + struct nfulnl_instance *inst = timer_container_of(inst, t, timer); spin_lock_bh(&inst->lock); if (inst->skb) diff --git a/net/netfilter/xt_IDLETIMER.c b/net/netfilter/xt_IDLETIMER.c index 9082155ee558..d73957592c9d 100644 --- a/net/netfilter/xt_IDLETIMER.c +++ b/net/netfilter/xt_IDLETIMER.c @@ -100,7 +100,7 @@ static void idletimer_tg_work(struct work_struct *work) static void idletimer_tg_expired(struct timer_list *t) { - struct idletimer_tg *timer = from_timer(timer, t, timer); + struct idletimer_tg *timer = timer_container_of(timer, t, timer); pr_debug("timer %s expired\n", timer->attr.attr.name); diff --git a/net/netfilter/xt_LED.c b/net/netfilter/xt_LED.c index 8a80fd76fe45..90dcf088071a 100644 --- a/net/netfilter/xt_LED.c +++ b/net/netfilter/xt_LED.c @@ -72,8 +72,9 @@ led_tg(struct sk_buff *skb, const struct xt_action_param *par) static void led_timeout_callback(struct timer_list *t) { - struct xt_led_info_internal *ledinternal = from_timer(ledinternal, t, - timer); + struct xt_led_info_internal *ledinternal = timer_container_of(ledinternal, + t, + timer); led_trigger_event(&ledinternal->netfilter_led_trigger, LED_OFF); } diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c index 6ee148f0e6d0..3331669d8e33 100644 --- a/net/netrom/af_netrom.c +++ b/net/netrom/af_netrom.c @@ -240,7 +240,7 @@ void nr_destroy_socket(struct sock *); */ static void nr_destroy_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); bh_lock_sock(sk); sock_hold(sk); nr_destroy_socket(sk); diff --git a/net/netrom/nr_timer.c b/net/netrom/nr_timer.c index 5e3ca068f04e..b3a62b1f3a09 100644 --- a/net/netrom/nr_timer.c +++ b/net/netrom/nr_timer.c @@ -111,7 +111,7 @@ int nr_t1timer_running(struct sock *sk) static void nr_heartbeat_expiry(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); struct nr_sock *nr = nr_sk(sk); bh_lock_sock(sk); @@ -152,7 +152,7 @@ out: static void nr_t2timer_expiry(struct timer_list *t) { - struct nr_sock *nr = from_timer(nr, t, t2timer); + struct nr_sock *nr = timer_container_of(nr, t, t2timer); struct sock *sk = &nr->sock; bh_lock_sock(sk); @@ -166,7 +166,7 @@ static void nr_t2timer_expiry(struct timer_list *t) static void nr_t4timer_expiry(struct timer_list *t) { - struct nr_sock *nr = from_timer(nr, t, t4timer); + struct nr_sock *nr = timer_container_of(nr, t, t4timer); struct sock *sk = &nr->sock; bh_lock_sock(sk); @@ -177,7 +177,7 @@ static void nr_t4timer_expiry(struct timer_list *t) static void nr_idletimer_expiry(struct timer_list *t) { - struct nr_sock *nr = from_timer(nr, t, idletimer); + struct nr_sock *nr = timer_container_of(nr, t, idletimer); struct sock *sk = &nr->sock; bh_lock_sock(sk); @@ -206,7 +206,7 @@ static void nr_idletimer_expiry(struct timer_list *t) static void nr_t1timer_expiry(struct timer_list *t) { - struct nr_sock *nr = from_timer(nr, t, t1timer); + struct nr_sock *nr = timer_container_of(nr, t, t1timer); struct sock *sk = &nr->sock; bh_lock_sock(sk); diff --git a/net/nfc/core.c b/net/nfc/core.c index 75ed8a9146ba..ae1c842f9c64 100644 --- a/net/nfc/core.c +++ b/net/nfc/core.c @@ -1010,7 +1010,7 @@ exit: static void nfc_check_pres_timeout(struct timer_list *t) { - struct nfc_dev *dev = from_timer(dev, t, check_pres_timer); + struct nfc_dev *dev = timer_container_of(dev, t, check_pres_timer); schedule_work(&dev->check_pres_work); } diff --git a/net/nfc/hci/core.c b/net/nfc/hci/core.c index aa493344d93e..8618d57c23da 100644 --- a/net/nfc/hci/core.c +++ b/net/nfc/hci/core.c @@ -441,7 +441,7 @@ exit_noskb: static void nfc_hci_cmd_timeout(struct timer_list *t) { - struct nfc_hci_dev *hdev = from_timer(hdev, t, cmd_timer); + struct nfc_hci_dev *hdev = timer_container_of(hdev, t, cmd_timer); schedule_work(&hdev->msg_tx_work); } diff --git a/net/nfc/hci/llc_shdlc.c b/net/nfc/hci/llc_shdlc.c index ce9c683a3ead..4fc37894860c 100644 --- a/net/nfc/hci/llc_shdlc.c +++ b/net/nfc/hci/llc_shdlc.c @@ -564,14 +564,14 @@ static void llc_shdlc_handle_send_queue(struct llc_shdlc *shdlc) static void llc_shdlc_connect_timeout(struct timer_list *t) { - struct llc_shdlc *shdlc = from_timer(shdlc, t, connect_timer); + struct llc_shdlc *shdlc = timer_container_of(shdlc, t, connect_timer); schedule_work(&shdlc->sm_work); } static void llc_shdlc_t1_timeout(struct timer_list *t) { - struct llc_shdlc *shdlc = from_timer(shdlc, t, t1_timer); + struct llc_shdlc *shdlc = timer_container_of(shdlc, t, t1_timer); pr_debug("SoftIRQ: need to send ack\n"); @@ -580,7 +580,7 @@ static void llc_shdlc_t1_timeout(struct timer_list *t) static void llc_shdlc_t2_timeout(struct timer_list *t) { - struct llc_shdlc *shdlc = from_timer(shdlc, t, t2_timer); + struct llc_shdlc *shdlc = timer_container_of(shdlc, t, t2_timer); pr_debug("SoftIRQ: need to retransmit\n"); diff --git a/net/nfc/llcp_core.c b/net/nfc/llcp_core.c index 27e863f96ed1..beeb3b4d28ca 100644 --- a/net/nfc/llcp_core.c +++ b/net/nfc/llcp_core.c @@ -243,7 +243,8 @@ static void nfc_llcp_timeout_work(struct work_struct *work) static void nfc_llcp_symm_timer(struct timer_list *t) { - struct nfc_llcp_local *local = from_timer(local, t, link_timer); + struct nfc_llcp_local *local = timer_container_of(local, t, + link_timer); pr_err("SYMM timeout\n"); @@ -286,7 +287,8 @@ static void nfc_llcp_sdreq_timeout_work(struct work_struct *work) static void nfc_llcp_sdreq_timer(struct timer_list *t) { - struct nfc_llcp_local *local = from_timer(local, t, sdreq_timer); + struct nfc_llcp_local *local = timer_container_of(local, t, + sdreq_timer); schedule_work(&local->sdreq_timeout_work); } diff --git a/net/nfc/nci/core.c b/net/nfc/nci/core.c index 0171bf3c7016..fc921cd2cdff 100644 --- a/net/nfc/nci/core.c +++ b/net/nfc/nci/core.c @@ -610,7 +610,7 @@ static int nci_close_device(struct nci_dev *ndev) /* NCI command timer function */ static void nci_cmd_timer(struct timer_list *t) { - struct nci_dev *ndev = from_timer(ndev, t, cmd_timer); + struct nci_dev *ndev = timer_container_of(ndev, t, cmd_timer); atomic_set(&ndev->cmd_cnt, 1); queue_work(ndev->cmd_wq, &ndev->cmd_work); @@ -619,7 +619,7 @@ static void nci_cmd_timer(struct timer_list *t) /* NCI data exchange timer function */ static void nci_data_timer(struct timer_list *t) { - struct nci_dev *ndev = from_timer(ndev, t, data_timer); + struct nci_dev *ndev = timer_container_of(ndev, t, data_timer); set_bit(NCI_DATA_EXCHANGE_TO, &ndev->flags); queue_work(ndev->rx_wq, &ndev->rx_work); diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 20be2c47cf41..3d43f3eae759 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -722,7 +722,7 @@ static void _prb_refresh_rx_retire_blk_timer(struct tpacket_kbdq_core *pkc) static void prb_retire_rx_blk_timer_expired(struct timer_list *t) { struct packet_sock *po = - from_timer(po, t, rx_ring.prb_bdqc.retire_blk_timer); + timer_container_of(po, t, rx_ring.prb_bdqc.retire_blk_timer); struct tpacket_kbdq_core *pkc = GET_PBDQC_FROM_RB(&po->rx_ring); unsigned int frozen; struct tpacket_block_desc *pbd; diff --git a/net/rose/af_rose.c b/net/rose/af_rose.c index a4a668b88a8f..4e72b636a46a 100644 --- a/net/rose/af_rose.c +++ b/net/rose/af_rose.c @@ -345,7 +345,7 @@ void rose_destroy_socket(struct sock *); */ static void rose_destroy_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); rose_destroy_socket(sk); } diff --git a/net/rose/rose_link.c b/net/rose/rose_link.c index 9f9629e6fdae..7746229fdc8c 100644 --- a/net/rose/rose_link.c +++ b/net/rose/rose_link.c @@ -78,7 +78,7 @@ static void rose_ftimer_expiry(struct timer_list *t) static void rose_t0timer_expiry(struct timer_list *t) { - struct rose_neigh *neigh = from_timer(neigh, t, t0timer); + struct rose_neigh *neigh = timer_container_of(neigh, t, t0timer); rose_transmit_restart_request(neigh); diff --git a/net/rose/rose_timer.c b/net/rose/rose_timer.c index 1525773e94aa..020369c49587 100644 --- a/net/rose/rose_timer.c +++ b/net/rose/rose_timer.c @@ -118,7 +118,7 @@ void rose_stop_idletimer(struct sock *sk) static void rose_heartbeat_expiry(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); struct rose_sock *rose = rose_sk(sk); bh_lock_sock(sk); @@ -163,7 +163,7 @@ out: static void rose_timer_expiry(struct timer_list *t) { - struct rose_sock *rose = from_timer(rose, t, timer); + struct rose_sock *rose = timer_container_of(rose, t, timer); struct sock *sk = &rose->sock; bh_lock_sock(sk); @@ -198,7 +198,7 @@ out: static void rose_idletimer_expiry(struct timer_list *t) { - struct rose_sock *rose = from_timer(rose, t, idletimer); + struct rose_sock *rose = timer_container_of(rose, t, idletimer); struct sock *sk = &rose->sock; bh_lock_sock(sk); diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c index e9e8f0ef3fd5..15067ff7b1f2 100644 --- a/net/rxrpc/call_object.c +++ b/net/rxrpc/call_object.c @@ -64,7 +64,7 @@ void rxrpc_poke_call(struct rxrpc_call *call, enum rxrpc_call_poke_trace what) static void rxrpc_call_timer_expired(struct timer_list *t) { - struct rxrpc_call *call = from_timer(call, t, timer); + struct rxrpc_call *call = timer_container_of(call, t, timer); _enter("%d", call->debug_id); diff --git a/net/sched/cls_flow.c b/net/sched/cls_flow.c index 5c2580a07530..5693b41b093f 100644 --- a/net/sched/cls_flow.c +++ b/net/sched/cls_flow.c @@ -345,7 +345,7 @@ TC_INDIRECT_SCOPE int flow_classify(struct sk_buff *skb, static void flow_perturbation(struct timer_list *t) { - struct flow_filter *f = from_timer(f, t, perturb_timer); + struct flow_filter *f = timer_container_of(f, t, perturb_timer); get_random_bytes(&f->hashrnd, 4); if (f->perturb_period) diff --git a/net/sched/sch_ets.c b/net/sched/sch_ets.c index 2c069f0181c6..037f764822b9 100644 --- a/net/sched/sch_ets.c +++ b/net/sched/sch_ets.c @@ -661,7 +661,7 @@ static int ets_qdisc_change(struct Qdisc *sch, struct nlattr *opt, for (i = q->nbands; i < oldbands; i++) { if (i >= q->nstrict && q->classes[i].qdisc->q.qlen) list_del_init(&q->classes[i].alist); - qdisc_tree_flush_backlog(q->classes[i].qdisc); + qdisc_purge_queue(q->classes[i].qdisc); } WRITE_ONCE(q->nstrict, nstrict); memcpy(q->prio2band, priomap, sizeof(priomap)); diff --git a/net/sched/sch_fq_pie.c b/net/sched/sch_fq_pie.c index df7fac95ab15..b0e34daf1f75 100644 --- a/net/sched/sch_fq_pie.c +++ b/net/sched/sch_fq_pie.c @@ -384,7 +384,7 @@ flow_error: static void fq_pie_timer(struct timer_list *t) { - struct fq_pie_sched_data *q = from_timer(q, t, adapt_timer); + struct fq_pie_sched_data *q = timer_container_of(q, t, adapt_timer); unsigned long next, tupdate; struct Qdisc *sch = q->sch; spinlock_t *root_lock; /* to lock qdisc for probability calculations */ diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 08e0e3aff976..16afb834fe4a 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -496,7 +496,7 @@ EXPORT_SYMBOL(netif_tx_unlock); static void dev_watchdog(struct timer_list *t) { - struct net_device *dev = from_timer(dev, t, watchdog_timer); + struct net_device *dev = timer_container_of(dev, t, watchdog_timer); bool release = true; spin_lock(&dev->tx_global_lock); diff --git a/net/sched/sch_pie.c b/net/sched/sch_pie.c index ff49a6c97033..ad46ee3ed5a9 100644 --- a/net/sched/sch_pie.c +++ b/net/sched/sch_pie.c @@ -424,7 +424,7 @@ EXPORT_SYMBOL_GPL(pie_calculate_probability); static void pie_timer(struct timer_list *t) { - struct pie_sched_data *q = from_timer(q, t, adapt_timer); + struct pie_sched_data *q = timer_container_of(q, t, adapt_timer); struct Qdisc *sch = q->sch; spinlock_t *root_lock; diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c index cc30f7a32f1a..9e2b9a490db2 100644 --- a/net/sched/sch_prio.c +++ b/net/sched/sch_prio.c @@ -211,7 +211,7 @@ static int prio_tune(struct Qdisc *sch, struct nlattr *opt, memcpy(q->prio2band, qopt->priomap, TC_PRIO_MAX+1); for (i = q->bands; i < oldbands; i++) - qdisc_tree_flush_backlog(q->queues[i]); + qdisc_purge_queue(q->queues[i]); for (i = oldbands; i < q->bands; i++) { q->queues[i] = queues[i]; diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c index 1ba3e0bba54f..479c42d11083 100644 --- a/net/sched/sch_red.c +++ b/net/sched/sch_red.c @@ -285,7 +285,7 @@ static int __red_change(struct Qdisc *sch, struct nlattr **tb, q->userbits = userbits; q->limit = ctl->limit; if (child) { - qdisc_tree_flush_backlog(q->qdisc); + qdisc_purge_queue(q->qdisc); old_child = q->qdisc; q->qdisc = child; } @@ -321,7 +321,7 @@ unlock_out: static inline void red_adaptative_timer(struct timer_list *t) { - struct red_sched_data *q = from_timer(q, t, adapt_timer); + struct red_sched_data *q = timer_container_of(q, t, adapt_timer); struct Qdisc *sch = q->sch; spinlock_t *root_lock; diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c index b912ad99aa15..96eb2f122973 100644 --- a/net/sched/sch_sfq.c +++ b/net/sched/sch_sfq.c @@ -310,7 +310,10 @@ drop: /* It is difficult to believe, but ALL THE SLOTS HAVE LENGTH 1. */ x = q->tail->next; slot = &q->slots[x]; - q->tail->next = slot->next; + if (slot->next == x) + q->tail = NULL; /* no more active slots */ + else + q->tail->next = slot->next; q->ht[slot->hash] = SFQ_EMPTY_SLOT; goto drop; } @@ -597,7 +600,7 @@ drop: static void sfq_perturbation(struct timer_list *t) { - struct sfq_sched_data *q = from_timer(q, t, perturb_timer); + struct sfq_sched_data *q = timer_container_of(q, t, perturb_timer); struct Qdisc *sch = q->sch; spinlock_t *root_lock; siphash_key_t nkey; @@ -653,6 +656,14 @@ static int sfq_change(struct Qdisc *sch, struct nlattr *opt, NL_SET_ERR_MSG_MOD(extack, "invalid quantum"); return -EINVAL; } + + if (ctl->perturb_period < 0 || + ctl->perturb_period > INT_MAX / HZ) { + NL_SET_ERR_MSG_MOD(extack, "invalid perturb period"); + return -EINVAL; + } + perturb_period = ctl->perturb_period * HZ; + if (ctl_v1 && !red_check_params(ctl_v1->qth_min, ctl_v1->qth_max, ctl_v1->Wlog, ctl_v1->Scell_log, NULL)) return -EINVAL; @@ -669,14 +680,12 @@ static int sfq_change(struct Qdisc *sch, struct nlattr *opt, headdrop = q->headdrop; maxdepth = q->maxdepth; maxflows = q->maxflows; - perturb_period = q->perturb_period; quantum = q->quantum; flags = q->flags; /* update and validate configuration */ if (ctl->quantum) quantum = ctl->quantum; - perturb_period = ctl->perturb_period * HZ; if (ctl->flows) maxflows = min_t(u32, ctl->flows, SFQ_MAX_FLOWS); if (ctl->divisor) { diff --git a/net/sched/sch_tbf.c b/net/sched/sch_tbf.c index dc26b22d53c7..4c977f049670 100644 --- a/net/sched/sch_tbf.c +++ b/net/sched/sch_tbf.c @@ -452,7 +452,7 @@ static int tbf_change(struct Qdisc *sch, struct nlattr *opt, sch_tree_lock(sch); if (child) { - qdisc_tree_flush_backlog(q->qdisc); + qdisc_purge_queue(q->qdisc); old = q->qdisc; q->qdisc = child; } diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c index 8c3b80c4d40b..f402f90eb6b6 100644 --- a/net/sctp/protocol.c +++ b/net/sctp/protocol.c @@ -631,7 +631,7 @@ static void sctp_v4_ecn_capable(struct sock *sk) static void sctp_addr_wq_timeout_handler(struct timer_list *t) { - struct net *net = from_timer(net, t, sctp.addr_wq_timer); + struct net *net = timer_container_of(net, t, sctp.addr_wq_timer); struct sctp_sockaddr_entry *addrw, *temp; struct sctp_sock *sp; diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c index 3aa5da5e3bbd..424f10a6fdba 100644 --- a/net/sctp/sm_sideeffect.c +++ b/net/sctp/sm_sideeffect.c @@ -231,7 +231,7 @@ nomem: void sctp_generate_t3_rtx_event(struct timer_list *t) { struct sctp_transport *transport = - from_timer(transport, t, T3_rtx_timer); + timer_container_of(transport, t, T3_rtx_timer); struct sctp_association *asoc = transport->asoc; struct sock *sk = asoc->base.sk; struct net *net = sock_net(sk); @@ -308,7 +308,8 @@ out_unlock: static void sctp_generate_t1_cookie_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_T1_COOKIE]); + timer_container_of(asoc, t, + timers[SCTP_EVENT_TIMEOUT_T1_COOKIE]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T1_COOKIE); } @@ -316,7 +317,8 @@ static void sctp_generate_t1_cookie_event(struct timer_list *t) static void sctp_generate_t1_init_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_T1_INIT]); + timer_container_of(asoc, t, + timers[SCTP_EVENT_TIMEOUT_T1_INIT]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T1_INIT); } @@ -324,7 +326,8 @@ static void sctp_generate_t1_init_event(struct timer_list *t) static void sctp_generate_t2_shutdown_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_T2_SHUTDOWN]); + timer_container_of(asoc, t, + timers[SCTP_EVENT_TIMEOUT_T2_SHUTDOWN]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T2_SHUTDOWN); } @@ -332,7 +335,7 @@ static void sctp_generate_t2_shutdown_event(struct timer_list *t) static void sctp_generate_t4_rto_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_T4_RTO]); + timer_container_of(asoc, t, timers[SCTP_EVENT_TIMEOUT_T4_RTO]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T4_RTO); } @@ -340,8 +343,8 @@ static void sctp_generate_t4_rto_event(struct timer_list *t) static void sctp_generate_t5_shutdown_guard_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, - timers[SCTP_EVENT_TIMEOUT_T5_SHUTDOWN_GUARD]); + timer_container_of(asoc, t, + timers[SCTP_EVENT_TIMEOUT_T5_SHUTDOWN_GUARD]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T5_SHUTDOWN_GUARD); @@ -351,7 +354,8 @@ static void sctp_generate_t5_shutdown_guard_event(struct timer_list *t) static void sctp_generate_autoclose_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_AUTOCLOSE]); + timer_container_of(asoc, t, + timers[SCTP_EVENT_TIMEOUT_AUTOCLOSE]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_AUTOCLOSE); } @@ -361,7 +365,8 @@ static void sctp_generate_autoclose_event(struct timer_list *t) */ void sctp_generate_heartbeat_event(struct timer_list *t) { - struct sctp_transport *transport = from_timer(transport, t, hb_timer); + struct sctp_transport *transport = timer_container_of(transport, t, + hb_timer); struct sctp_association *asoc = transport->asoc; struct sock *sk = asoc->base.sk; struct net *net = sock_net(sk); @@ -407,7 +412,7 @@ out_unlock: void sctp_generate_proto_unreach_event(struct timer_list *t) { struct sctp_transport *transport = - from_timer(transport, t, proto_unreach_timer); + timer_container_of(transport, t, proto_unreach_timer); struct sctp_association *asoc = transport->asoc; struct sock *sk = asoc->base.sk; struct net *net = sock_net(sk); @@ -442,7 +447,7 @@ out_unlock: void sctp_generate_reconf_event(struct timer_list *t) { struct sctp_transport *transport = - from_timer(transport, t, reconf_timer); + timer_container_of(transport, t, reconf_timer); struct sctp_association *asoc = transport->asoc; struct sock *sk = asoc->base.sk; struct net *net = sock_net(sk); @@ -478,7 +483,8 @@ out_unlock: /* Handle the timeout of the probe timer. */ void sctp_generate_probe_event(struct timer_list *t) { - struct sctp_transport *transport = from_timer(transport, t, probe_timer); + struct sctp_transport *transport = timer_container_of(transport, t, + probe_timer); struct sctp_association *asoc = transport->asoc; struct sock *sk = asoc->base.sk; struct net *net = sock_net(sk); @@ -511,7 +517,7 @@ out_unlock: static void sctp_generate_sack_event(struct timer_list *t) { struct sctp_association *asoc = - from_timer(asoc, t, timers[SCTP_EVENT_TIMEOUT_SACK]); + timer_container_of(asoc, t, timers[SCTP_EVENT_TIMEOUT_SACK]); sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_SACK); } diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index cb14d6ddac6c..8b1837228799 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -922,7 +922,7 @@ void svc_send(struct svc_rqst *rqstp) */ static void svc_age_temp_xprts(struct timer_list *t) { - struct svc_serv *serv = from_timer(serv, t, sv_temptimer); + struct svc_serv *serv = timer_container_of(serv, t, sv_temptimer); struct svc_xprt *xprt; struct list_head *le, *next; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index d5e0cdcad9e0..1023361845f9 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -854,7 +854,7 @@ xprt_schedule_autodisconnect(struct rpc_xprt *xprt) static void xprt_init_autodisconnect(struct timer_list *t) { - struct rpc_xprt *xprt = from_timer(xprt, t, timer); + struct rpc_xprt *xprt = timer_container_of(xprt, t, timer); if (!RB_EMPTY_ROOT(&xprt->recv_queue)) return; diff --git a/net/tipc/discover.c b/net/tipc/discover.c index 685389d4b245..775fd4f3f072 100644 --- a/net/tipc/discover.c +++ b/net/tipc/discover.c @@ -292,7 +292,7 @@ void tipc_disc_remove_dest(struct tipc_discoverer *d) */ static void tipc_disc_timeout(struct timer_list *t) { - struct tipc_discoverer *d = from_timer(d, t, timer); + struct tipc_discoverer *d = timer_container_of(d, t, timer); struct tipc_net *tn = tipc_net(d->net); struct tipc_media_addr maddr; struct sk_buff *skb = NULL; diff --git a/net/tipc/monitor.c b/net/tipc/monitor.c index b45c5b91bc7a..572b79bf76ce 100644 --- a/net/tipc/monitor.c +++ b/net/tipc/monitor.c @@ -630,7 +630,7 @@ void tipc_mon_get_state(struct net *net, u32 addr, static void mon_timeout(struct timer_list *t) { - struct tipc_monitor *mon = from_timer(mon, t, timer); + struct tipc_monitor *mon = timer_container_of(mon, t, timer); struct tipc_peer *self; int best_member_cnt = dom_size(mon->peer_cnt) - 1; diff --git a/net/tipc/node.c b/net/tipc/node.c index cb43f2016a70..a07fb073368c 100644 --- a/net/tipc/node.c +++ b/net/tipc/node.c @@ -800,7 +800,7 @@ static bool tipc_node_cleanup(struct tipc_node *peer) */ static void tipc_node_timeout(struct timer_list *t) { - struct tipc_node *n = from_timer(n, t, timer); + struct tipc_node *n = timer_container_of(n, t, timer); struct tipc_link_entry *le; struct sk_buff_head xmitq; int remains = n->link_cnt; diff --git a/net/tipc/socket.c b/net/tipc/socket.c index 65dcbb54f55d..7c61d47ea208 100644 --- a/net/tipc/socket.c +++ b/net/tipc/socket.c @@ -2862,7 +2862,7 @@ static void tipc_sk_retry_connect(struct sock *sk, struct sk_buff_head *list) static void tipc_sk_timeout(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); struct tipc_sock *tsk = tipc_sk(sk); u32 pnode = tsk_peer_node(tsk); struct sk_buff_head list; diff --git a/net/tipc/subscr.c b/net/tipc/subscr.c index 621addab2834..f8490d94e323 100644 --- a/net/tipc/subscr.c +++ b/net/tipc/subscr.c @@ -105,7 +105,7 @@ void tipc_sub_report_overlap(struct tipc_subscription *sub, static void tipc_sub_timeout(struct timer_list *t) { - struct tipc_subscription *sub = from_timer(sub, t, timer); + struct tipc_subscription *sub = timer_container_of(sub, t, timer); spin_lock(&sub->lock); tipc_sub_send_event(sub, NULL, TIPC_SUBSCR_TIMEOUT); diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 2e2e9997a68e..22e170fb5dda 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1971,7 +1971,8 @@ static void unix_maybe_add_creds(struct sk_buff *skb, const struct sock *sk, if (UNIXCB(skb).pid) return; - if (unix_may_passcred(sk) || unix_may_passcred(other)) { + if (unix_may_passcred(sk) || unix_may_passcred(other) || + !other->sk_socket) { UNIXCB(skb).pid = get_pid(task_tgid(current)); current_uid_gid(&UNIXCB(skb).uid, &UNIXCB(skb).gid); } diff --git a/net/wireless/core.c b/net/wireless/core.c index dcce326fdb8c..5c3c72df0591 100644 --- a/net/wireless/core.c +++ b/net/wireless/core.c @@ -1709,7 +1709,7 @@ EXPORT_SYMBOL_GPL(wiphy_work_flush); void wiphy_delayed_work_timer(struct timer_list *t) { - struct wiphy_delayed_work *dwork = from_timer(dwork, t, timer); + struct wiphy_delayed_work *dwork = timer_container_of(dwork, t, timer); wiphy_work_queue(dwork->wiphy, &dwork->work); } diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index fd5f79266471..85f139016da2 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -1583,7 +1583,7 @@ nl80211_parse_connkeys(struct cfg80211_registered_device *rdev, return result; error: - kfree(result); + kfree_sensitive(result); return ERR_PTR(err); } diff --git a/net/x25/af_x25.c b/net/x25/af_x25.c index 8dda4178497c..1f8ae9f4a3f1 100644 --- a/net/x25/af_x25.c +++ b/net/x25/af_x25.c @@ -359,7 +359,7 @@ static void __x25_destroy_socket(struct sock *); */ static void x25_destroy_timer(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); x25_destroy_socket_from_timer(sk); } diff --git a/net/x25/x25_link.c b/net/x25/x25_link.c index 37b190499405..4608aa5b4f31 100644 --- a/net/x25/x25_link.c +++ b/net/x25/x25_link.c @@ -46,7 +46,7 @@ static inline void x25_start_t20timer(struct x25_neigh *nb) static void x25_t20timer_expiry(struct timer_list *t) { - struct x25_neigh *nb = from_timer(nb, t, t20timer); + struct x25_neigh *nb = timer_container_of(nb, t, t20timer); x25_transmit_restart_request(nb); diff --git a/net/x25/x25_timer.c b/net/x25/x25_timer.c index e4c5ad5b070f..2ec63a1f4c6d 100644 --- a/net/x25/x25_timer.c +++ b/net/x25/x25_timer.c @@ -89,7 +89,7 @@ unsigned long x25_display_timer(struct sock *sk) static void x25_heartbeat_expiry(struct timer_list *t) { - struct sock *sk = from_timer(sk, t, sk_timer); + struct sock *sk = timer_container_of(sk, t, sk_timer); bh_lock_sock(sk); if (sock_owned_by_user(sk)) /* can currently only occur in state 3 */ @@ -156,7 +156,7 @@ static inline void x25_do_timer_expiry(struct sock * sk) static void x25_timer_expiry(struct timer_list *t) { - struct x25_sock *x25 = from_timer(x25, t, timer); + struct x25_sock *x25 = timer_container_of(x25, t, timer); struct sock *sk = &x25->sk; bh_lock_sock(sk); diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index d4134a18c658..094d2454602e 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -353,7 +353,7 @@ static inline unsigned long make_jiffies(long secs) static void xfrm_policy_timer(struct timer_list *t) { - struct xfrm_policy *xp = from_timer(xp, t, timer); + struct xfrm_policy *xp = timer_container_of(xp, t, timer); time64_t now = ktime_get_real_seconds(); time64_t next = TIME64_MAX; int warn = 0; @@ -2898,7 +2898,7 @@ static void xfrm_policy_queue_process(struct timer_list *t) struct sk_buff *skb; struct sock *sk; struct dst_entry *dst; - struct xfrm_policy *pol = from_timer(pol, t, polq.hold_timer); + struct xfrm_policy *pol = timer_container_of(pol, t, polq.hold_timer); struct net *net = xp_net(pol); struct xfrm_policy_queue *pq = &pol->polq; struct flowi fl; diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c index 203b585c2ae2..77cc418ad69e 100644 --- a/net/xfrm/xfrm_state.c +++ b/net/xfrm/xfrm_state.c @@ -2697,7 +2697,7 @@ EXPORT_SYMBOL(xfrm_state_walk_done); static void xfrm_replay_timer_handler(struct timer_list *t) { - struct xfrm_state *x = from_timer(x, t, rtimer); + struct xfrm_state *x = timer_container_of(x, t, rtimer); spin_lock(&x->lock); diff --git a/rust/helpers/cpu.c b/rust/helpers/cpu.c new file mode 100644 index 000000000000..824e0adb19d4 --- /dev/null +++ b/rust/helpers/cpu.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/smp.h> + +unsigned int rust_helper_raw_smp_processor_id(void) +{ + return raw_smp_processor_id(); +} diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index 0f1b5d115985..16fa9bca5949 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -13,6 +13,7 @@ #include "build_assert.c" #include "build_bug.c" #include "clk.c" +#include "cpu.c" #include "cpufreq.c" #include "cpumask.c" #include "cred.c" diff --git a/rust/kernel/cpu.rs b/rust/kernel/cpu.rs index 10c5c3b25873..b75403b0eb56 100644 --- a/rust/kernel/cpu.rs +++ b/rust/kernel/cpu.rs @@ -6,6 +6,127 @@ use crate::{bindings, device::Device, error::Result, prelude::ENODEV}; +/// Returns the maximum number of possible CPUs in the current system configuration. +#[inline] +pub fn nr_cpu_ids() -> u32 { + #[cfg(any(NR_CPUS_1, CONFIG_FORCE_NR_CPUS))] + { + bindings::NR_CPUS + } + + #[cfg(not(any(NR_CPUS_1, CONFIG_FORCE_NR_CPUS)))] + // SAFETY: `nr_cpu_ids` is a valid global provided by the kernel. + unsafe { + bindings::nr_cpu_ids + } +} + +/// The CPU ID. +/// +/// Represents a CPU identifier as a wrapper around an [`u32`]. +/// +/// # Invariants +/// +/// The CPU ID lies within the range `[0, nr_cpu_ids())`. +/// +/// # Examples +/// +/// ``` +/// use kernel::cpu::CpuId; +/// +/// let cpu = 0; +/// +/// // SAFETY: 0 is always a valid CPU number. +/// let id = unsafe { CpuId::from_u32_unchecked(cpu) }; +/// +/// assert_eq!(id.as_u32(), cpu); +/// assert!(CpuId::from_i32(0).is_some()); +/// assert!(CpuId::from_i32(-1).is_none()); +/// ``` +#[derive(Copy, Clone, PartialEq, Eq, Debug)] +pub struct CpuId(u32); + +impl CpuId { + /// Creates a new [`CpuId`] from the given `id` without checking bounds. + /// + /// # Safety + /// + /// The caller must ensure that `id` is a valid CPU ID (i.e., `0 <= id < nr_cpu_ids()`). + #[inline] + pub unsafe fn from_i32_unchecked(id: i32) -> Self { + debug_assert!(id >= 0); + debug_assert!((id as u32) < nr_cpu_ids()); + + // INVARIANT: The function safety guarantees `id` is a valid CPU id. + Self(id as u32) + } + + /// Creates a new [`CpuId`] from the given `id`, checking that it is valid. + pub fn from_i32(id: i32) -> Option<Self> { + if id < 0 || id as u32 >= nr_cpu_ids() { + None + } else { + // INVARIANT: `id` has just been checked as a valid CPU ID. + Some(Self(id as u32)) + } + } + + /// Creates a new [`CpuId`] from the given `id` without checking bounds. + /// + /// # Safety + /// + /// The caller must ensure that `id` is a valid CPU ID (i.e., `0 <= id < nr_cpu_ids()`). + #[inline] + pub unsafe fn from_u32_unchecked(id: u32) -> Self { + debug_assert!(id < nr_cpu_ids()); + + // Ensure the `id` fits in an [`i32`] as it's also representable that way. + debug_assert!(id <= i32::MAX as u32); + + // INVARIANT: The function safety guarantees `id` is a valid CPU id. + Self(id) + } + + /// Creates a new [`CpuId`] from the given `id`, checking that it is valid. + pub fn from_u32(id: u32) -> Option<Self> { + if id >= nr_cpu_ids() { + None + } else { + // INVARIANT: `id` has just been checked as a valid CPU ID. + Some(Self(id)) + } + } + + /// Returns CPU number. + #[inline] + pub fn as_u32(&self) -> u32 { + self.0 + } + + /// Returns the ID of the CPU the code is currently running on. + /// + /// The returned value is considered unstable because it may change + /// unexpectedly due to preemption or CPU migration. It should only be + /// used when the context ensures that the task remains on the same CPU + /// or the users could use a stale (yet valid) CPU ID. + pub fn current() -> Self { + // SAFETY: raw_smp_processor_id() always returns a valid CPU ID. + unsafe { Self::from_u32_unchecked(bindings::raw_smp_processor_id()) } + } +} + +impl From<CpuId> for u32 { + fn from(id: CpuId) -> Self { + id.as_u32() + } +} + +impl From<CpuId> for i32 { + fn from(id: CpuId) -> Self { + id.as_u32() as i32 + } +} + /// Creates a new instance of CPU's device. /// /// # Safety @@ -17,9 +138,9 @@ use crate::{bindings, device::Device, error::Result, prelude::ENODEV}; /// Callers must ensure that the CPU device is not used after it has been unregistered. /// This can be achieved, for example, by registering a CPU hotplug notifier and removing /// any references to the CPU device within the notifier's callback. -pub unsafe fn from_cpu(cpu: u32) -> Result<&'static Device> { +pub unsafe fn from_cpu(cpu: CpuId) -> Result<&'static Device> { // SAFETY: It is safe to call `get_cpu_device()` for any CPU. - let ptr = unsafe { bindings::get_cpu_device(cpu) }; + let ptr = unsafe { bindings::get_cpu_device(u32::from(cpu)) }; if ptr.is_null() { return Err(ENODEV); } diff --git a/rust/kernel/cpufreq.rs b/rust/kernel/cpufreq.rs index b0a9c6182aec..11b03e9d7e89 100644 --- a/rust/kernel/cpufreq.rs +++ b/rust/kernel/cpufreq.rs @@ -10,6 +10,7 @@ use crate::{ clk::Hertz, + cpu::CpuId, cpumask, device::{Bound, Device}, devres::Devres, @@ -465,8 +466,9 @@ impl Policy { /// Returns the primary CPU for the [`Policy`]. #[inline] - pub fn cpu(&self) -> u32 { - self.as_ref().cpu + pub fn cpu(&self) -> CpuId { + // SAFETY: The C API guarantees that `cpu` refers to a valid CPU number. + unsafe { CpuId::from_u32_unchecked(self.as_ref().cpu) } } /// Returns the minimum frequency for the [`Policy`]. @@ -525,7 +527,7 @@ impl Policy { #[inline] pub fn generic_get(&self) -> Result<u32> { // SAFETY: By the type invariant, the pointer stored in `self` is valid. - Ok(unsafe { bindings::cpufreq_generic_get(self.cpu()) }) + Ok(unsafe { bindings::cpufreq_generic_get(u32::from(self.cpu())) }) } /// Provides a wrapper to the register with energy model using the OPP core. @@ -678,9 +680,9 @@ impl Policy { struct PolicyCpu<'a>(&'a mut Policy); impl<'a> PolicyCpu<'a> { - fn from_cpu(cpu: u32) -> Result<Self> { + fn from_cpu(cpu: CpuId) -> Result<Self> { // SAFETY: It is safe to call `cpufreq_cpu_get` for any valid CPU. - let ptr = from_err_ptr(unsafe { bindings::cpufreq_cpu_get(cpu) })?; + let ptr = from_err_ptr(unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) })?; Ok(Self( // SAFETY: The `ptr` is guaranteed to be valid and remains valid for the lifetime of @@ -1055,8 +1057,11 @@ impl<T: Driver> Registration<T> { impl<T: Driver> Registration<T> { /// Driver's `init` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn init_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn init_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1070,8 +1075,11 @@ impl<T: Driver> Registration<T> { /// Driver's `exit` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn exit_callback(ptr: *mut bindings::cpufreq_policy) { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn exit_callback(ptr: *mut bindings::cpufreq_policy) { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. let policy = unsafe { Policy::from_raw_mut(ptr) }; @@ -1082,8 +1090,11 @@ impl<T: Driver> Registration<T> { /// Driver's `online` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn online_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn online_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1094,8 +1105,13 @@ impl<T: Driver> Registration<T> { /// Driver's `offline` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn offline_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn offline_callback( + ptr: *mut bindings::cpufreq_policy, + ) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1106,8 +1122,13 @@ impl<T: Driver> Registration<T> { /// Driver's `suspend` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn suspend_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn suspend_callback( + ptr: *mut bindings::cpufreq_policy, + ) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1118,8 +1139,11 @@ impl<T: Driver> Registration<T> { /// Driver's `resume` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn resume_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn resume_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1130,8 +1154,11 @@ impl<T: Driver> Registration<T> { /// Driver's `ready` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn ready_callback(ptr: *mut bindings::cpufreq_policy) { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn ready_callback(ptr: *mut bindings::cpufreq_policy) { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. let policy = unsafe { Policy::from_raw_mut(ptr) }; @@ -1140,8 +1167,13 @@ impl<T: Driver> Registration<T> { /// Driver's `verify` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn verify_callback(ptr: *mut bindings::cpufreq_policy_data) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn verify_callback( + ptr: *mut bindings::cpufreq_policy_data, + ) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1152,8 +1184,13 @@ impl<T: Driver> Registration<T> { /// Driver's `setpolicy` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn setpolicy_callback(ptr: *mut bindings::cpufreq_policy) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn setpolicy_callback( + ptr: *mut bindings::cpufreq_policy, + ) -> kernel::ffi::c_int { from_result(|| { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. @@ -1164,8 +1201,11 @@ impl<T: Driver> Registration<T> { /// Driver's `target` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn target_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn target_callback( ptr: *mut bindings::cpufreq_policy, target_freq: u32, relation: u32, @@ -1180,8 +1220,11 @@ impl<T: Driver> Registration<T> { /// Driver's `target_index` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn target_index_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn target_index_callback( ptr: *mut bindings::cpufreq_policy, index: u32, ) -> kernel::ffi::c_int { @@ -1200,8 +1243,11 @@ impl<T: Driver> Registration<T> { /// Driver's `fast_switch` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn fast_switch_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn fast_switch_callback( ptr: *mut bindings::cpufreq_policy, target_freq: u32, ) -> kernel::ffi::c_uint { @@ -1212,21 +1258,31 @@ impl<T: Driver> Registration<T> { } /// Driver's `adjust_perf` callback. - extern "C" fn adjust_perf_callback( + /// + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + unsafe extern "C" fn adjust_perf_callback( cpu: u32, min_perf: usize, target_perf: usize, capacity: usize, ) { - if let Ok(mut policy) = PolicyCpu::from_cpu(cpu) { + // SAFETY: The C API guarantees that `cpu` refers to a valid CPU number. + let cpu_id = unsafe { CpuId::from_u32_unchecked(cpu) }; + + if let Ok(mut policy) = PolicyCpu::from_cpu(cpu_id) { T::adjust_perf(&mut policy, min_perf, target_perf, capacity); } } /// Driver's `get_intermediate` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn get_intermediate_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn get_intermediate_callback( ptr: *mut bindings::cpufreq_policy, index: u32, ) -> kernel::ffi::c_uint { @@ -1243,8 +1299,11 @@ impl<T: Driver> Registration<T> { /// Driver's `target_intermediate` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn target_intermediate_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn target_intermediate_callback( ptr: *mut bindings::cpufreq_policy, index: u32, ) -> kernel::ffi::c_int { @@ -1262,12 +1321,24 @@ impl<T: Driver> Registration<T> { } /// Driver's `get` callback. - extern "C" fn get_callback(cpu: u32) -> kernel::ffi::c_uint { - PolicyCpu::from_cpu(cpu).map_or(0, |mut policy| T::get(&mut policy).map_or(0, |f| f)) + /// + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + unsafe extern "C" fn get_callback(cpu: u32) -> kernel::ffi::c_uint { + // SAFETY: The C API guarantees that `cpu` refers to a valid CPU number. + let cpu_id = unsafe { CpuId::from_u32_unchecked(cpu) }; + + PolicyCpu::from_cpu(cpu_id).map_or(0, |mut policy| T::get(&mut policy).map_or(0, |f| f)) } /// Driver's `update_limit` callback. - extern "C" fn update_limits_callback(ptr: *mut bindings::cpufreq_policy) { + /// + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn update_limits_callback(ptr: *mut bindings::cpufreq_policy) { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. let policy = unsafe { Policy::from_raw_mut(ptr) }; @@ -1276,10 +1347,16 @@ impl<T: Driver> Registration<T> { /// Driver's `bios_limit` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn bios_limit_callback(cpu: i32, limit: *mut u32) -> kernel::ffi::c_int { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn bios_limit_callback(cpu: i32, limit: *mut u32) -> kernel::ffi::c_int { + // SAFETY: The C API guarantees that `cpu` refers to a valid CPU number. + let cpu_id = unsafe { CpuId::from_i32_unchecked(cpu) }; + from_result(|| { - let mut policy = PolicyCpu::from_cpu(cpu as u32)?; + let mut policy = PolicyCpu::from_cpu(cpu_id)?; // SAFETY: `limit` is guaranteed by the C code to be valid. T::bios_limit(&mut policy, &mut (unsafe { *limit })).map(|()| 0) @@ -1288,8 +1365,11 @@ impl<T: Driver> Registration<T> { /// Driver's `set_boost` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn set_boost_callback( + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn set_boost_callback( ptr: *mut bindings::cpufreq_policy, state: i32, ) -> kernel::ffi::c_int { @@ -1303,8 +1383,11 @@ impl<T: Driver> Registration<T> { /// Driver's `register_em` callback. /// - /// SAFETY: Called from C. Inputs must be valid pointers. - extern "C" fn register_em_callback(ptr: *mut bindings::cpufreq_policy) { + /// # Safety + /// + /// - This function may only be called from the cpufreq C infrastructure. + /// - The pointer arguments must be valid pointers. + unsafe extern "C" fn register_em_callback(ptr: *mut bindings::cpufreq_policy) { // SAFETY: The `ptr` is guaranteed to be valid by the contract with the C code for the // lifetime of `policy`. let policy = unsafe { Policy::from_raw_mut(ptr) }; diff --git a/rust/kernel/cpumask.rs b/rust/kernel/cpumask.rs index c90bfac9346a..19c607709b5f 100644 --- a/rust/kernel/cpumask.rs +++ b/rust/kernel/cpumask.rs @@ -6,6 +6,7 @@ use crate::{ alloc::{AllocError, Flags}, + cpu::CpuId, prelude::*, types::Opaque, }; @@ -35,9 +36,10 @@ use core::ops::{Deref, DerefMut}; /// /// ``` /// use kernel::bindings; +/// use kernel::cpu::CpuId; /// use kernel::cpumask::Cpumask; /// -/// fn set_clear_cpu(ptr: *mut bindings::cpumask, set_cpu: u32, clear_cpu: i32) { +/// fn set_clear_cpu(ptr: *mut bindings::cpumask, set_cpu: CpuId, clear_cpu: CpuId) { /// // SAFETY: The `ptr` is valid for writing and remains valid for the lifetime of the /// // returned reference. /// let mask = unsafe { Cpumask::as_mut_ref(ptr) }; @@ -90,9 +92,9 @@ impl Cpumask { /// This mismatches kernel naming convention and corresponds to the C /// function `__cpumask_set_cpu()`. #[inline] - pub fn set(&mut self, cpu: u32) { + pub fn set(&mut self, cpu: CpuId) { // SAFETY: By the type invariant, `self.as_raw` is a valid argument to `__cpumask_set_cpu`. - unsafe { bindings::__cpumask_set_cpu(cpu, self.as_raw()) }; + unsafe { bindings::__cpumask_set_cpu(u32::from(cpu), self.as_raw()) }; } /// Clear `cpu` in the cpumask. @@ -101,19 +103,19 @@ impl Cpumask { /// This mismatches kernel naming convention and corresponds to the C /// function `__cpumask_clear_cpu()`. #[inline] - pub fn clear(&mut self, cpu: i32) { + pub fn clear(&mut self, cpu: CpuId) { // SAFETY: By the type invariant, `self.as_raw` is a valid argument to // `__cpumask_clear_cpu`. - unsafe { bindings::__cpumask_clear_cpu(cpu, self.as_raw()) }; + unsafe { bindings::__cpumask_clear_cpu(i32::from(cpu), self.as_raw()) }; } /// Test `cpu` in the cpumask. /// /// Equivalent to the kernel's `cpumask_test_cpu` API. #[inline] - pub fn test(&self, cpu: i32) -> bool { + pub fn test(&self, cpu: CpuId) -> bool { // SAFETY: By the type invariant, `self.as_raw` is a valid argument to `cpumask_test_cpu`. - unsafe { bindings::cpumask_test_cpu(cpu, self.as_raw()) } + unsafe { bindings::cpumask_test_cpu(i32::from(cpu), self.as_raw()) } } /// Set all CPUs in the cpumask. @@ -178,21 +180,40 @@ impl Cpumask { /// The following example demonstrates how to create and update a [`CpumaskVar`]. /// /// ``` +/// use kernel::cpu::CpuId; /// use kernel::cpumask::CpumaskVar; /// /// let mut mask = CpumaskVar::new_zero(GFP_KERNEL).unwrap(); /// /// assert!(mask.empty()); -/// mask.set(2); -/// assert!(mask.test(2)); -/// mask.set(3); -/// assert!(mask.test(3)); -/// assert_eq!(mask.weight(), 2); +/// let mut count = 0; +/// +/// let cpu2 = CpuId::from_u32(2); +/// if let Some(cpu) = cpu2 { +/// mask.set(cpu); +/// assert!(mask.test(cpu)); +/// count += 1; +/// } +/// +/// let cpu3 = CpuId::from_u32(3); +/// if let Some(cpu) = cpu3 { +/// mask.set(cpu); +/// assert!(mask.test(cpu)); +/// count += 1; +/// } +/// +/// assert_eq!(mask.weight(), count); /// /// let mask2 = CpumaskVar::try_clone(&mask).unwrap(); -/// assert!(mask2.test(2)); -/// assert!(mask2.test(3)); -/// assert_eq!(mask2.weight(), 2); +/// +/// if let Some(cpu) = cpu2 { +/// assert!(mask2.test(cpu)); +/// } +/// +/// if let Some(cpu) = cpu3 { +/// assert!(mask2.test(cpu)); +/// } +/// assert_eq!(mask2.weight(), count); /// ``` pub struct CpumaskVar { #[cfg(CONFIG_CPUMASK_OFFSTACK)] diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 557e725ab932..a6461ea411f7 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -37,6 +37,90 @@ include $(srctree)/scripts/Makefile.compiler include $(kbuild-file) include $(srctree)/scripts/Makefile.lib +# flags that take effect in current and sub directories +KBUILD_AFLAGS += $(subdir-asflags-y) +KBUILD_CFLAGS += $(subdir-ccflags-y) +KBUILD_RUSTFLAGS += $(subdir-rustflags-y) + +# Figure out what we need to build from the various variables +# =========================================================================== + +# When an object is listed to be built compiled-in and modular, +# only build the compiled-in version +obj-m := $(filter-out $(obj-y),$(obj-m)) + +# Libraries are always collected in one lib file. +# Filter out objects already built-in +lib-y := $(filter-out $(obj-y), $(sort $(lib-y) $(lib-m))) + +# Subdirectories we need to descend into +subdir-ym := $(sort $(subdir-y) $(subdir-m) \ + $(patsubst %/,%, $(filter %/, $(obj-y) $(obj-m)))) + +# Handle objects in subdirs: +# - If we encounter foo/ in $(obj-y), replace it by foo/built-in.a and +# foo/modules.order +# - If we encounter foo/ in $(obj-m), replace it by foo/modules.order +# +# Generate modules.order to determine modorder. Unfortunately, we don't have +# information about ordering between -y and -m subdirs. Just put -y's first. + +ifdef need-modorder +obj-m := $(patsubst %/,%/modules.order, $(filter %/, $(obj-y)) $(obj-m)) +else +obj-m := $(filter-out %/, $(obj-m)) +endif + +ifdef need-builtin +obj-y := $(patsubst %/, %/built-in.a, $(obj-y)) +else +obj-y := $(filter-out %/, $(obj-y)) +endif + +# Expand $(foo-objs) $(foo-y) etc. by replacing their individuals +suffix-search = $(strip $(foreach s, $3, $($(1:%$(strip $2)=%$s)))) +# List composite targets that are constructed by combining other targets +multi-search = $(sort $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $m))) +# List primitive targets that are compiled from source files +real-search = $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $(call suffix-search, $m, $2, $3), $m)) + +# If $(foo-objs), $(foo-y), $(foo-m), or $(foo-) exists, foo.o is a composite object +multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y) +multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m) +multi-obj-ym := $(multi-obj-y) $(multi-obj-m) + +# Replace multi-part objects by their individual parts, +# including built-in.a from subdirectories +real-obj-y := $(call real-search, $(obj-y), .o, -objs -y) +real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m) + +always-y += $(always-m) + +# hostprogs-always-y += foo +# ... is a shorthand for +# hostprogs += foo +# always-y += foo +hostprogs += $(hostprogs-always-y) $(hostprogs-always-m) +always-y += $(hostprogs-always-y) $(hostprogs-always-m) + +# userprogs-always-y is likewise. +userprogs += $(userprogs-always-y) $(userprogs-always-m) +always-y += $(userprogs-always-y) $(userprogs-always-m) + +# Add subdir path + +ifneq ($(obj),.) +extra-y := $(addprefix $(obj)/, $(extra-y)) +always-y := $(addprefix $(obj)/, $(always-y)) +targets := $(addprefix $(obj)/, $(targets)) +obj-m := $(addprefix $(obj)/, $(obj-m)) +lib-y := $(addprefix $(obj)/, $(lib-y)) +real-obj-y := $(addprefix $(obj)/, $(real-obj-y)) +real-obj-m := $(addprefix $(obj)/, $(real-obj-m)) +multi-obj-m := $(addprefix $(obj)/, $(multi-obj-m)) +subdir-ym := $(addprefix $(obj)/, $(subdir-ym)) +endif + ifndef obj $(warning kbuild: Makefile.build is included improperly) endif diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 2b332645e0c2..1d581ba5df66 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -1,89 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -# flags that take effect in current and sub directories -KBUILD_AFLAGS += $(subdir-asflags-y) -KBUILD_CFLAGS += $(subdir-ccflags-y) -KBUILD_RUSTFLAGS += $(subdir-rustflags-y) - -# Figure out what we need to build from the various variables -# =========================================================================== - -# When an object is listed to be built compiled-in and modular, -# only build the compiled-in version -obj-m := $(filter-out $(obj-y),$(obj-m)) - -# Libraries are always collected in one lib file. -# Filter out objects already built-in -lib-y := $(filter-out $(obj-y), $(sort $(lib-y) $(lib-m))) - -# Subdirectories we need to descend into -subdir-ym := $(sort $(subdir-y) $(subdir-m) \ - $(patsubst %/,%, $(filter %/, $(obj-y) $(obj-m)))) - -# Handle objects in subdirs: -# - If we encounter foo/ in $(obj-y), replace it by foo/built-in.a and -# foo/modules.order -# - If we encounter foo/ in $(obj-m), replace it by foo/modules.order -# -# Generate modules.order to determine modorder. Unfortunately, we don't have -# information about ordering between -y and -m subdirs. Just put -y's first. - -ifdef need-modorder -obj-m := $(patsubst %/,%/modules.order, $(filter %/, $(obj-y)) $(obj-m)) -else -obj-m := $(filter-out %/, $(obj-m)) -endif - -ifdef need-builtin -obj-y := $(patsubst %/, %/built-in.a, $(obj-y)) -else -obj-y := $(filter-out %/, $(obj-y)) -endif - -# Expand $(foo-objs) $(foo-y) etc. by replacing their individuals -suffix-search = $(strip $(foreach s, $3, $($(1:%$(strip $2)=%$s)))) -# List composite targets that are constructed by combining other targets -multi-search = $(sort $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $m))) -# List primitive targets that are compiled from source files -real-search = $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $(call suffix-search, $m, $2, $3), $m)) - -# If $(foo-objs), $(foo-y), $(foo-m), or $(foo-) exists, foo.o is a composite object -multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y) -multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m) -multi-obj-ym := $(multi-obj-y) $(multi-obj-m) - -# Replace multi-part objects by their individual parts, -# including built-in.a from subdirectories -real-obj-y := $(call real-search, $(obj-y), .o, -objs -y) -real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m) - -always-y += $(always-m) - -# hostprogs-always-y += foo -# ... is a shorthand for -# hostprogs += foo -# always-y += foo -hostprogs += $(hostprogs-always-y) $(hostprogs-always-m) -always-y += $(hostprogs-always-y) $(hostprogs-always-m) - -# userprogs-always-y is likewise. -userprogs += $(userprogs-always-y) $(userprogs-always-m) -always-y += $(userprogs-always-y) $(userprogs-always-m) - -# Add subdir path - -ifneq ($(obj),.) -extra-y := $(addprefix $(obj)/,$(extra-y)) -always-y := $(addprefix $(obj)/,$(always-y)) -targets := $(addprefix $(obj)/,$(targets)) -obj-m := $(addprefix $(obj)/,$(obj-m)) -lib-y := $(addprefix $(obj)/,$(lib-y)) -real-obj-y := $(addprefix $(obj)/,$(real-obj-y)) -real-obj-m := $(addprefix $(obj)/,$(real-obj-m)) -multi-obj-m := $(addprefix $(obj)/, $(multi-obj-m)) -subdir-ym := $(addprefix $(obj)/,$(subdir-ym)) -endif - # Finds the multi-part object the current object will be linked into. # If the object belongs to two or more multi-part objects, list them all. modname-multi = $(sort $(foreach m,$(multi-obj-ym),\ diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index eed247d8abfc..13ea7bf1ae7d 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -228,12 +228,24 @@ static void process_fqn(struct die *cache, Dwarf_Die *die) DEFINE_PROCESS_UDATA_ATTRIBUTE(accessibility) DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) DEFINE_PROCESS_UDATA_ATTRIBUTE(bit_size) -DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) DEFINE_PROCESS_UDATA_ATTRIBUTE(data_bit_offset) DEFINE_PROCESS_UDATA_ATTRIBUTE(data_member_location) DEFINE_PROCESS_UDATA_ATTRIBUTE(discr_value) +static void process_byte_size_attr(struct die *cache, Dwarf_Die *die) +{ + Dwarf_Word value; + unsigned long override; + + if (get_udata_attr(die, DW_AT_byte_size, &value)) { + if (stable && kabi_get_byte_size(cache->fqn, &override)) + value = override; + + process_fmt(cache, " byte_size(%" PRIu64 ")", value); + } +} + /* Match functions -- die_match_callback_t */ #define DEFINE_MATCH(type) \ static bool match_##type##_type(Dwarf_Die *die) \ diff --git a/scripts/gendwarfksyms/examples/kabi.h b/scripts/gendwarfksyms/examples/kabi.h index 97a5669b083d..170733a3fba4 100644 --- a/scripts/gendwarfksyms/examples/kabi.h +++ b/scripts/gendwarfksyms/examples/kabi.h @@ -37,11 +37,14 @@ #define __stringify(x...) __stringify_1(x) #endif -#define __KABI_RULE(hint, target, value) \ +#define ___KABI_RULE(hint, target, value) \ static const char __PASTE(__gendwarfksyms_rule_, \ __COUNTER__)[] __used __aligned(1) \ __section(".discard.gendwarfksyms.kabi_rules") = \ - "1\0" #hint "\0" #target "\0" #value + "1\0" #hint "\0" target "\0" value + +#define __KABI_RULE(hint, target, value) \ + ___KABI_RULE(hint, #target, #value) #define __KABI_NORMAL_SIZE_ALIGN(_orig, _new) \ union { \ @@ -90,6 +93,20 @@ __KABI_RULE(enumerator_value, fqn field, value) /* + * KABI_BYTE_SIZE(fqn, value) + * Set the byte_size attribute for the struct/union/enum fqn to + * value bytes. + */ +#define KABI_BYTE_SIZE(fqn, value) __KABI_RULE(byte_size, fqn, value) + +/* + * KABI_TYPE_STRING(type, str) + * For the given type, override the type string used in symtypes + * output and version calculation with str. + */ +#define KABI_TYPE_STRING(type, str) ___KABI_RULE(type_string, type, str) + +/* * KABI_RESERVE * Reserve some "padding" in a structure for use by LTS backports. * This is normally placed at the end of a structure. diff --git a/scripts/gendwarfksyms/examples/kabi_ex.c b/scripts/gendwarfksyms/examples/kabi_ex.c index 0b7ffd830541..1f799eb7c756 100644 --- a/scripts/gendwarfksyms/examples/kabi_ex.c +++ b/scripts/gendwarfksyms/examples/kabi_ex.c @@ -28,3 +28,10 @@ struct ex2c ex2c; struct ex3a ex3a; struct ex3b ex3b; struct ex3c ex3c; + +struct ex4a ex4a; + +struct ex5a ex5a; +struct ex5b ex5b; + +int ex6a; diff --git a/scripts/gendwarfksyms/examples/kabi_ex.h b/scripts/gendwarfksyms/examples/kabi_ex.h index 1736e0f65208..785b211d9c58 100644 --- a/scripts/gendwarfksyms/examples/kabi_ex.h +++ b/scripts/gendwarfksyms/examples/kabi_ex.h @@ -21,6 +21,12 @@ * ./gendwarfksyms --stable --dump-dies \ * examples/kabi_ex.o 2>&1 >/dev/null | \ * FileCheck examples/kabi_ex.h --check-prefix=STABLE + + * $ nm examples/kabi_ex.o | awk '{ print $NF }' | \ + * ./gendwarfksyms --stable --dump-versions \ + * examples/kabi_ex.o 2>&1 >/dev/null | \ + * sort | \ + * FileCheck examples/kabi_ex.h --check-prefix=VERSIONS */ #ifndef __KABI_EX_H__ @@ -170,7 +176,7 @@ struct ex2a { /* * STABLE: variable structure_type ex2a { * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , - * STABLE-NEXT: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) b data_member_location(8) + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) b data_member_location(8) * STABLE-NEXT: member base_type int byte_size(4) encoding(5) c data_member_location(16) , * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) d data_member_location(24) * STABLE-NEXT: } byte_size(32) @@ -227,7 +233,7 @@ struct ex3a { /* * STABLE: variable structure_type ex3a { - * STABLE-NEXT: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) a data_member_location(0) + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) a data_member_location(0) * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) unused data_member_location(8) * STABLE-NEXT: } byte_size(16) */ @@ -260,4 +266,95 @@ _Static_assert(sizeof(struct ex3a) == sizeof(struct ex3c), "ex3a size doesn't ma * STABLE-NEXT: } byte_size(16) */ +/* + * Example: An ignored field added to an end of a partially opaque struct, + * while keeping the byte_size attribute unchanged. + */ + +struct ex4a { + unsigned long a; + KABI_IGNORE(0, unsigned long b); +}; + +/* + * This may be safe if the structure allocation is managed by the core kernel + * and the layout remains unchanged except for appended new members. + */ +KABI_BYTE_SIZE(ex4a, 8); + +/* + * STABLE: variable structure_type ex4a { + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) a data_member_location(0) + * STABLE-NEXT: } byte_size(8) + */ + +/* + * Example: A type string override. + */ + +struct ex5a { + unsigned long a; +}; + +/* + * This may be safe if the structure is fully opaque to modules, even though + * its definition has inadvertently become part of the ABI. + */ +KABI_TYPE_STRING( + "s#ex5a", + "structure_type ex5a { member pointer_type { s#ex4a } byte_size(8) p data_member_location(0) } byte_size(8)"); + +/* + * Make sure the fully expanded type string includes ex4a. + * + * VERSIONS: ex5a variable structure_type ex5a { + * VERSIONS-SAME: member pointer_type { + * VERSIONS-SAME: structure_type ex4a { + * VERSIONS-SAME: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) a data_member_location(0) + * VERSIONS-SAME: } byte_size(8) + * VERSIONS-SAME: } byte_size(8) p data_member_location(0) + * VERSIONS-SAME: } byte_size(8) + */ + +/* + * Example: A type string definition for a non-existent type. + */ + +struct ex5b { + unsigned long a; +}; + +/* Replace the type string for struct ex5b */ +KABI_TYPE_STRING( + "s#ex5b", + "structure_type ex5b { member pointer_type { s#ex5c } byte_size(8) p data_member_location(0) } byte_size(8)"); + +/* Define a type string for a non-existent struct ex5c */ +KABI_TYPE_STRING( + "s#ex5c", + "structure_type ex5c { member base_type int byte_size(4) encoding(5) n data_member_location(0) } byte_size(8)"); + +/* + * Make sure the fully expanded type string includes the definition for ex5c. + * + * VERSIONS: ex5b variable structure_type ex5b { + * VERSIONS-SAME: member pointer_type { + * VERSIONS-SAME: structure_type ex5c { + * VERSIONS-SAME: member base_type int byte_size(4) encoding(5) n data_member_location(0) + * VERSIONS-SAME: } byte_size(8) + * VERSIONS-SAME: } byte_size(8) p data_member_location(0) + * VERSIONS-SAME: } byte_size(8) + */ + +/* + * Example: A type string override for a symbol. + */ + +KABI_TYPE_STRING("ex6a", "variable s#ex5c"); + +/* + * VERSIONS: ex6a variable structure_type ex5c { + * VERSIONS-SAME: member base_type int byte_size(4) encoding(5) n data_member_location(0) + * VERSIONS-SAME: } byte_size(8) + */ #endif /* __KABI_EX_H__ */ diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 2feec168bf73..7dd03ffe0c5c 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -287,10 +287,12 @@ void generate_symtypes_and_versions(FILE *file); * kabi.c */ +bool kabi_get_byte_size(const char *fqn, unsigned long *value); bool kabi_is_enumerator_ignored(const char *fqn, const char *field); bool kabi_get_enumerator_value(const char *fqn, const char *field, unsigned long *value); bool kabi_is_declonly(const char *fqn); +bool kabi_get_type_string(const char *type, const char **str); void kabi_read_rules(int fd); void kabi_free(void); diff --git a/scripts/gendwarfksyms/kabi.c b/scripts/gendwarfksyms/kabi.c index 66f01fcd1607..b3ade713778f 100644 --- a/scripts/gendwarfksyms/kabi.c +++ b/scripts/gendwarfksyms/kabi.c @@ -54,11 +54,27 @@ */ #define KABI_RULE_TAG_ENUMERATOR_VALUE "enumerator_value" +/* + * Rule: byte_size + * - For the fqn_field in the target field, set the byte_size + * attribute to the value in the value field. + */ +#define KABI_RULE_TAG_BYTE_SIZE "byte_size" + +/* + * Rule: type_string + * - For the type reference in the fqn field, use the type string + * in the value field. + */ +#define KABI_RULE_TAG_TYPE_STRING "type_string" + enum kabi_rule_type { KABI_RULE_TYPE_UNKNOWN, KABI_RULE_TYPE_DECLONLY, KABI_RULE_TYPE_ENUMERATOR_IGNORE, KABI_RULE_TYPE_ENUMERATOR_VALUE, + KABI_RULE_TYPE_BYTE_SIZE, + KABI_RULE_TYPE_TYPE_STRING, }; #define RULE_HASH_BITS 7 @@ -127,6 +143,14 @@ void kabi_read_rules(int fd) .type = KABI_RULE_TYPE_ENUMERATOR_VALUE, .tag = KABI_RULE_TAG_ENUMERATOR_VALUE, }, + { + .type = KABI_RULE_TYPE_BYTE_SIZE, + .tag = KABI_RULE_TAG_BYTE_SIZE, + }, + { + .type = KABI_RULE_TYPE_TYPE_STRING, + .tag = KABI_RULE_TAG_TYPE_STRING, + }, }; if (!stable) @@ -222,33 +246,55 @@ void kabi_read_rules(int fd) check(elf_end(elf)); } -bool kabi_is_declonly(const char *fqn) +static char *get_enumerator_target(const char *fqn, const char *field) +{ + char *target = NULL; + + if (asprintf(&target, "%s %s", fqn, field) < 0) + error("asprintf failed for '%s %s'", fqn, field); + + return target; +} + +static struct rule *find_rule(enum kabi_rule_type type, const char *target) { struct rule *rule; if (!stable) - return false; - if (!fqn || !*fqn) - return false; + return NULL; + if (!target || !*target) + return NULL; hash_for_each_possible(rules, rule, hash, - rule_values_hash(KABI_RULE_TYPE_DECLONLY, fqn)) { - if (rule->type == KABI_RULE_TYPE_DECLONLY && - !strcmp(fqn, rule->target)) - return true; + rule_values_hash(type, target)) { + if (rule->type == type && !strcmp(target, rule->target)) + return rule; } - return false; + return NULL; } -static char *get_enumerator_target(const char *fqn, const char *field) +static struct rule *find_enumerator_rule(enum kabi_rule_type type, + const char *fqn, const char *field) { - char *target = NULL; + struct rule *rule; + char *target; - if (asprintf(&target, "%s %s", fqn, field) < 0) - error("asprintf failed for '%s %s'", fqn, field); + if (!stable) + return NULL; + if (!fqn || !*fqn || !field || !*field) + return NULL; - return target; + target = get_enumerator_target(fqn, field); + rule = find_rule(type, target); + + free(target); + return rule; +} + +bool kabi_is_declonly(const char *fqn) +{ + return !!find_rule(KABI_RULE_TYPE_DECLONLY, fqn); } static unsigned long get_ulong_value(const char *value) @@ -267,58 +313,49 @@ static unsigned long get_ulong_value(const char *value) bool kabi_is_enumerator_ignored(const char *fqn, const char *field) { - bool match = false; - struct rule *rule; - char *target; - - if (!stable) - return false; - if (!fqn || !*fqn || !field || !*field) - return false; + return !!find_enumerator_rule(KABI_RULE_TYPE_ENUMERATOR_IGNORE, fqn, + field); +} - target = get_enumerator_target(fqn, field); +bool kabi_get_enumerator_value(const char *fqn, const char *field, + unsigned long *value) +{ + struct rule *rule; - hash_for_each_possible( - rules, rule, hash, - rule_values_hash(KABI_RULE_TYPE_ENUMERATOR_IGNORE, target)) { - if (rule->type == KABI_RULE_TYPE_ENUMERATOR_IGNORE && - !strcmp(target, rule->target)) { - match = true; - break; - } + rule = find_enumerator_rule(KABI_RULE_TYPE_ENUMERATOR_VALUE, fqn, + field); + if (rule) { + *value = get_ulong_value(rule->value); + return true; } - free(target); - return match; + return false; } -bool kabi_get_enumerator_value(const char *fqn, const char *field, - unsigned long *value) +bool kabi_get_byte_size(const char *fqn, unsigned long *value) { - bool match = false; struct rule *rule; - char *target; - if (!stable) - return false; - if (!fqn || !*fqn || !field || !*field) - return false; + rule = find_rule(KABI_RULE_TYPE_BYTE_SIZE, fqn); + if (rule) { + *value = get_ulong_value(rule->value); + return true; + } - target = get_enumerator_target(fqn, field); + return false; +} - hash_for_each_possible(rules, rule, hash, - rule_values_hash(KABI_RULE_TYPE_ENUMERATOR_VALUE, - target)) { - if (rule->type == KABI_RULE_TYPE_ENUMERATOR_VALUE && - !strcmp(target, rule->target)) { - *value = get_ulong_value(rule->value); - match = true; - break; - } +bool kabi_get_type_string(const char *type, const char **str) +{ + struct rule *rule; + + rule = find_rule(KABI_RULE_TYPE_TYPE_STRING, type); + if (rule) { + *str = rule->value; + return true; } - free(target); - return match; + return false; } void kabi_free(void) diff --git a/scripts/gendwarfksyms/types.c b/scripts/gendwarfksyms/types.c index 6f37289104ff..39ce1770e463 100644 --- a/scripts/gendwarfksyms/types.c +++ b/scripts/gendwarfksyms/types.c @@ -100,7 +100,7 @@ static void type_expansion_append(struct type_expansion *type, const char *s, #define TYPE_HASH_BITS 12 static HASHTABLE_DEFINE(type_map, 1 << TYPE_HASH_BITS); -static int type_map_get(const char *name, struct type_expansion **res) +static int __type_map_get(const char *name, struct type_expansion **res) { struct type_expansion *e; @@ -114,11 +114,12 @@ static int type_map_get(const char *name, struct type_expansion **res) return -1; } -static void type_map_add(const char *name, struct type_expansion *type) +static struct type_expansion *type_map_add(const char *name, + struct type_expansion *type) { struct type_expansion *e; - if (type_map_get(name, &e)) { + if (__type_map_get(name, &e)) { e = xmalloc(sizeof(struct type_expansion)); type_expansion_init(e); e->name = xstrdup(name); @@ -130,7 +131,7 @@ static void type_map_add(const char *name, struct type_expansion *type) } else { /* Use the longest available expansion */ if (type->len <= e->len) - return; + return e; type_list_free(&e->expanded); @@ -148,6 +149,34 @@ static void type_map_add(const char *name, struct type_expansion *type) type_list_write(&e->expanded, stderr); checkp(fputs("\n", stderr)); } + + return e; +} + +static void type_parse(const char *name, const char *str, + struct type_expansion *type); + +static int type_map_get(const char *name, struct type_expansion **res) +{ + struct type_expansion type; + const char *override; + + if (!__type_map_get(name, res)) + return 0; + + /* + * If die_map didn't contain a type, we might still have + * a type_string kABI rule that defines it. + */ + if (stable && kabi_get_type_string(name, &override)) { + type_expansion_init(&type); + type_parse(name, override, &type); + *res = type_map_add(name, &type); + type_expansion_free(&type); + return 0; + } + + return -1; } static void type_map_write(FILE *file) @@ -267,15 +296,18 @@ static char *get_type_name(struct die *cache) return name; } -static void __calculate_version(struct version *version, struct list_head *list) +static void __calculate_version(struct version *version, + struct type_expansion *type) { struct type_list_entry *entry; struct type_expansion *e; /* Calculate a CRC over an expanded type string */ - list_for_each_entry(entry, list, list) { + list_for_each_entry(entry, &type->expanded, list) { if (is_type_prefix(entry->str)) { - check(type_map_get(entry->str, &e)); + if (type_map_get(entry->str, &e)) + error("unknown type reference to '%s' when expanding '%s'", + entry->str, type->name); /* * It's sufficient to expand each type reference just @@ -285,7 +317,7 @@ static void __calculate_version(struct version *version, struct list_head *list) version_add(version, entry->str); } else { cache_mark_expanded(&expansion_cache, e); - __calculate_version(version, &e->expanded); + __calculate_version(version, e); } } else { version_add(version, entry->str); @@ -293,10 +325,11 @@ static void __calculate_version(struct version *version, struct list_head *list) } } -static void calculate_version(struct version *version, struct list_head *list) +static void calculate_version(struct version *version, + struct type_expansion *type) { version_init(version); - __calculate_version(version, list); + __calculate_version(version, type); cache_free(&expansion_cache); } @@ -372,9 +405,80 @@ static void type_expand(struct die *cache, struct type_expansion *type, cache_free(&expansion_cache); } +static void type_parse(const char *name, const char *str, + struct type_expansion *type) +{ + char *fragment; + size_t start = 0; + size_t end; + size_t pos; + + if (!*str) + error("empty type string override for '%s'", name); + + type_expansion_init(type); + + for (pos = 0; str[pos]; ++pos) { + bool empty; + char marker = ' '; + + if (!is_type_prefix(&str[pos])) + continue; + + end = pos + 2; + + /* + * Find the end of the type reference. If the type name contains + * spaces, it must be in single quotes. + */ + if (str[end] == '\'') { + marker = '\''; + ++end; + } + while (str[end] && str[end] != marker) + ++end; + + /* Check that we have a non-empty type name */ + if (marker == '\'') { + if (str[end] != marker) + error("incomplete %c# type reference for '%s' (string : '%s')", + str[pos], name, str); + empty = end == pos + 3; + ++end; + } else { + empty = end == pos + 2; + } + if (empty) + error("empty %c# type name for '%s' (string: '%s')", + str[pos], name, str); + + /* Append the part of the string before the type reference */ + if (pos > start) { + fragment = xstrndup(&str[start], pos - start); + type_expansion_append(type, fragment, fragment); + } + + /* + * Append the type reference -- note that if the reference + * is invalid, i.e. points to a non-existent type, we will + * print out an error when calculating versions. + */ + fragment = xstrndup(&str[pos], end - pos); + type_expansion_append(type, fragment, fragment); + + start = end; + pos = end - 1; + } + + /* Append the rest of the type string, if there's any left */ + if (str[start]) + type_expansion_append(type, &str[start], NULL); +} + static void expand_type(struct die *cache, void *arg) { struct type_expansion type; + const char *override; char *name; if (cache->mapped) @@ -399,9 +503,13 @@ static void expand_type(struct die *cache, void *arg) return; debug("%s", name); - type_expand(cache, &type, true); - type_map_add(name, &type); + if (stable && kabi_get_type_string(name, &override)) + type_parse(name, override, &type); + else + type_expand(cache, &type, true); + + type_map_add(name, &type); type_expansion_free(&type); free(name); } @@ -410,6 +518,7 @@ static void expand_symbol(struct symbol *sym, void *arg) { struct type_expansion type; struct version version; + const char *override; struct die *cache; /* @@ -423,11 +532,14 @@ static void expand_symbol(struct symbol *sym, void *arg) if (__die_map_get(sym->die_addr, DIE_SYMBOL, &cache)) return; /* We'll warn about missing CRCs later. */ - type_expand(cache, &type, false); + if (stable && kabi_get_type_string(sym->name, &override)) + type_parse(sym->name, override, &type); + else + type_expand(cache, &type, false); /* If the symbol already has a version, don't calculate it again. */ if (sym->state != SYMBOL_PROCESSED) { - calculate_version(&version, &type.expanded); + calculate_version(&version, &type); symbol_set_crc(sym, version.crc); debug("%s = %lx", sym->name, version.crc); diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index 8b0d7ac73dbb..83e48670c2fc 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -181,13 +181,9 @@ static int is_unknown_symbol(struct symbol *sym) strcmp(defn->string, "{") == 0); } -static struct symbol *__add_symbol(const char *name, enum symbol_type type, - struct string_list *defn, int is_extern, - int is_reference) +static struct string_list *process_enum(const char *name, enum symbol_type type, + struct string_list *defn) { - unsigned long h; - struct symbol *sym; - enum symbol_status status = STATUS_UNCHANGED; /* The parser adds symbols in the order their declaration completes, * so it is safe to store the value of the previous enum constant in * a static variable. @@ -216,7 +212,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, defn = mk_node(buf); } } - } else if (type == SYM_ENUM) { + } else { free_list(last_enum_expr, NULL); last_enum_expr = NULL; enum_counter = 0; @@ -225,6 +221,23 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, return NULL; } + return defn; +} + +static struct symbol *__add_symbol(const char *name, enum symbol_type type, + struct string_list *defn, int is_extern, + int is_reference) +{ + unsigned long h; + struct symbol *sym; + enum symbol_status status = STATUS_UNCHANGED; + + if ((type == SYM_ENUM_CONST || type == SYM_ENUM) && !is_reference) { + defn = process_enum(name, type, defn); + if (defn == NULL) + return NULL; + } + h = crc32(name); hash_for_each_possible(symbol_hashtable, sym, hnode, h) { if (map_to_ns(sym->type) != map_to_ns(type) || diff --git a/scripts/kconfig/expr.h b/scripts/kconfig/expr.h index 21578dcd4292..fe2231e0e6a4 100644 --- a/scripts/kconfig/expr.h +++ b/scripts/kconfig/expr.h @@ -205,15 +205,26 @@ struct property { for (st = sym->prop; st; st = st->next) \ if (st->text) +enum menu_type { + M_CHOICE, // "choice" + M_COMMENT, // "comment" + M_IF, // "if" + M_MENU, // "mainmenu", "menu", "menuconfig" + M_NORMAL, // others, i.e., "config" +}; + /* * Represents a node in the menu tree, as seen in e.g. menuconfig (though used * for all front ends). Each symbol, menu, etc. defined in the Kconfig files * gets a node. A symbol defined in multiple locations gets one node at each * location. * + * @type: type of the menu entry * @choice_members: list of choice members with priority. */ struct menu { + enum menu_type type; + /* The next menu node at the same level */ struct menu *next; diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h index b8ebc3094a23..fbc907f75eac 100644 --- a/scripts/kconfig/lkc.h +++ b/scripts/kconfig/lkc.h @@ -81,7 +81,7 @@ void _menu_init(void); void menu_warn(const struct menu *menu, const char *fmt, ...); struct menu *menu_add_menu(void); void menu_end_menu(void); -void menu_add_entry(struct symbol *sym); +void menu_add_entry(struct symbol *sym, enum menu_type type); void menu_add_dep(struct expr *dep); void menu_add_visibility(struct expr *dep); struct property *menu_add_prompt(enum prop_type type, const char *prompt, diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c index 6587ac86d0d5..7d48a692bd27 100644 --- a/scripts/kconfig/menu.c +++ b/scripts/kconfig/menu.c @@ -15,7 +15,7 @@ static const char nohelp_text[] = "There is no help available for this option."; -struct menu rootmenu; +struct menu rootmenu = { .type = M_MENU }; static struct menu **last_entry_ptr; /** @@ -65,12 +65,13 @@ void _menu_init(void) last_entry_ptr = &rootmenu.list; } -void menu_add_entry(struct symbol *sym) +void menu_add_entry(struct symbol *sym, enum menu_type type) { struct menu *menu; menu = xmalloc(sizeof(*menu)); memset(menu, 0, sizeof(*menu)); + menu->type = type; menu->sym = sym; menu->parent = current_menu; menu->filename = cur_filename; diff --git a/scripts/kconfig/parser.y b/scripts/kconfig/parser.y index 68372d3ff325..e9c3c664e925 100644 --- a/scripts/kconfig/parser.y +++ b/scripts/kconfig/parser.y @@ -139,7 +139,7 @@ stmt_list_in_choice: config_entry_start: T_CONFIG nonconst_symbol T_EOL { - menu_add_entry($2); + menu_add_entry($2, M_NORMAL); printd(DEBUG_PARSE, "%s:%d:config %s\n", cur_filename, cur_lineno, $2->name); }; @@ -173,7 +173,7 @@ config_stmt: config_entry_start config_option_list menuconfig_entry_start: T_MENUCONFIG nonconst_symbol T_EOL { - menu_add_entry($2); + menu_add_entry($2, M_MENU); printd(DEBUG_PARSE, "%s:%d:menuconfig %s\n", cur_filename, cur_lineno, $2->name); }; @@ -246,7 +246,7 @@ choice: T_CHOICE T_EOL { struct symbol *sym = sym_lookup(NULL, 0); - menu_add_entry(sym); + menu_add_entry(sym, M_CHOICE); menu_set_type(S_BOOLEAN); INIT_LIST_HEAD(¤t_entry->choice_members); @@ -315,7 +315,7 @@ default: if_entry: T_IF expr T_EOL { printd(DEBUG_PARSE, "%s:%d:if\n", cur_filename, cur_lineno); - menu_add_entry(NULL); + menu_add_entry(NULL, M_IF); menu_add_dep($2); $$ = menu_add_menu(); }; @@ -338,7 +338,7 @@ if_stmt_in_choice: if_entry stmt_list_in_choice if_end menu: T_MENU T_WORD_QUOTE T_EOL { - menu_add_entry(NULL); + menu_add_entry(NULL, M_MENU); menu_add_prompt(P_MENU, $2, NULL); printd(DEBUG_PARSE, "%s:%d:menu\n", cur_filename, cur_lineno); }; @@ -376,7 +376,7 @@ source_stmt: T_SOURCE T_WORD_QUOTE T_EOL comment: T_COMMENT T_WORD_QUOTE T_EOL { - menu_add_entry(NULL); + menu_add_entry(NULL, M_COMMENT); menu_add_prompt(P_COMMENT, $2, NULL); printd(DEBUG_PARSE, "%s:%d:comment\n", cur_filename, cur_lineno); }; diff --git a/scripts/misc-check b/scripts/misc-check index d40d5484e0c5..a74450e799d1 100755 --- a/scripts/misc-check +++ b/scripts/misc-check @@ -3,17 +3,65 @@ set -e -# Detect files that are tracked but ignored by git. This is checked only when -# ${KBUILD_EXTRA_WARN} contains 1, git is installed, and the source tree is -# tracked by git. +# Detect files that are tracked but ignored by git. check_tracked_ignored_files () { - case "${KBUILD_EXTRA_WARN}" in - *1*) ;; - *) return;; - esac - - git -C ${srctree:-.} ls-files -i -c --exclude-per-directory=.gitignore 2>/dev/null | + git -C "${srctree:-.}" ls-files -i -c --exclude-per-directory=.gitignore 2>/dev/null | sed 's/$/: warning: ignored by one of the .gitignore files/' >&2 } +# Check for missing #include <linux/export.h> +# +# The rule for including <linux/export.h> is very simple: +# Include <linux/export.h> only when you use EXPORT_SYMBOL(). That's it. +# +# However, some headers include <linux/export.h> even though they are completely +# unrelated to EXPORT_SYMBOL(). +# +# One example is include/linux/module.h. Please note <linux/module.h> and +# <linux/export.h> are orthogonal. <linux/module.h> should be included by files +# that can be compiled as modules. In other words, <linux/module.h> should be +# included by EXPORT_SYMBOL consumers. In contrast, <linux/export.h> should be +# included from EXPORT_SYMBOL providers, which may or may not be modular. +# Hence, include/linux/module.h should *not* include <linux/export.h>. +# +# Another example is include/linux/linkage.h, which is completely unrelated to +# EXPORT_SYMBOL(). Worse, it is included by most C files, which means, most C +# files end up including <linux/export.h>, even though only some of them +# actually export symbols. Hence, include/linux/linkage.h should *not* include +# <linux/export.h>. +# +# Before fixing such headers, we must ensure that C files using EXPORT_SYMBOL() +# include <linux/export.h> directly, since many C files currently rely on +# <linux/export.h> being included indirectly (likely, via <linux/linkage> etc.). +# +# Therefore, this check. +# +# The problem is simple - the warned files use EXPORT_SYMBOL(), but do not +# include <linux/export.h>. Please add #include <linux/export.h> to them. +# +# If the included headers are sorted alphabetically, please insert +# <linux/export.h> in the appropriate position to maintain the sort order. +# For this reason, this script only checks missing <linux/export.h>, but +# does not automatically fix it. +check_missing_include_linux_export_h () { + + git -C "${srctree:-.}" grep --files-with-matches -E 'EXPORT_SYMBOL((_NS)?(_GPL)?|_GPL_FOR_MODULES)\(.*\)' \ + -- '*.[ch]' :^tools/ :^include/linux/export.h | + xargs -r git -C "${srctree:-.}" grep --files-without-match '#include[[:space:]]*<linux/export\.h>' | + xargs -r printf "%s: warning: EXPORT_SYMBOL() is used, but #include <linux/export.h> is missing\n" >&2 +} + +# If you do not use EXPORT_SYMBOL(), please do not include <linux/export.h>. +# Currently, this is checked for *.c files, but not for *.h files, because some +# *.c files rely on <linux/export.h> being included indirectly. +check_unnecessary_include_linux_export_h () { + + git -C "${srctree:-.}" grep --files-with-matches '#include[[:space:]]*<linux/export\.h>' \ + -- '*.[c]' :^tools/ | + xargs -r git -C "${srctree:-.}" grep --files-without-match -E 'EXPORT_SYMBOL((_NS)?(_GPL)?|_GPL_FOR_MODULES)\(.*\)' | + xargs -r printf "%s: warning: EXPORT_SYMBOL() is not used, but #include <linux/export.h> is present\n" >&2 +} + check_tracked_ignored_files +check_missing_include_linux_export_h +check_unnecessary_include_linux_export_h diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index be89921d60b6..5ca7c268294e 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -28,6 +28,8 @@ #include "modpost.h" #include "../../include/linux/license.h" +#define MODULE_NS_PREFIX "module:" + static bool module_enabled; /* Are we using CONFIG_MODVERSIONS? */ static bool modversions; @@ -1595,11 +1597,14 @@ static void read_symbols(const char *modname) license = get_next_modinfo(&info, "license", license); } - namespace = get_modinfo(&info, "import_ns"); - while (namespace) { + for (namespace = get_modinfo(&info, "import_ns"); + namespace; + namespace = get_next_modinfo(&info, "import_ns", namespace)) { + if (strstarts(namespace, MODULE_NS_PREFIX)) + error("%s: explicitly importing namespace \"%s\" is not allowed.\n", + mod->name, namespace); + add_namespace(&mod->imported_namespaces, namespace); - namespace = get_next_modinfo(&info, "import_ns", - namespace); } if (!get_modinfo(&info, "description")) @@ -1684,6 +1689,46 @@ void buf_write(struct buffer *buf, const char *s, int len) buf->pos += len; } +/** + * verify_module_namespace() - does @modname have access to this symbol's @namespace + * @namespace: export symbol namespace + * @modname: module name + * + * If @namespace is prefixed with "module:" to indicate it is a module namespace + * then test if @modname matches any of the comma separated patterns. + * + * The patterns only support tail-glob. + */ +static bool verify_module_namespace(const char *namespace, const char *modname) +{ + size_t len, modlen = strlen(modname); + const char *prefix = "module:"; + const char *sep; + bool glob; + + if (!strstarts(namespace, prefix)) + return false; + + for (namespace += strlen(prefix); *namespace; namespace = sep) { + sep = strchrnul(namespace, ','); + len = sep - namespace; + + glob = false; + if (sep[-1] == '*') { + len--; + glob = true; + } + + if (*sep) + sep++; + + if (strncmp(namespace, modname, len) == 0 && (glob || len == modlen)) + return true; + } + + return false; +} + static void check_exports(struct module *mod) { struct symbol *s, *exp; @@ -1711,7 +1756,8 @@ static void check_exports(struct module *mod) basename = get_basename(mod->name); - if (!contains_namespace(&mod->imported_namespaces, exp->namespace)) { + if (!verify_module_namespace(exp->namespace, basename) && + !contains_namespace(&mod->imported_namespaces, exp->namespace)) { modpost_log(!allow_missing_ns_imports, "module %s uses symbol %s from namespace %s, but does not import it.\n", basename, exp->name, exp->namespace); diff --git a/scripts/tags.sh b/scripts/tags.sh index 98680e9cd7be..99ce427d9a69 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -344,7 +344,7 @@ case "$1" in "tags") rm -f tags - xtags ctags + xtags ${CTAGS:-ctags} remove_structs=y ;; diff --git a/security/keys/gc.c b/security/keys/gc.c index f27223ea4578..748e83818a76 100644 --- a/security/keys/gc.c +++ b/security/keys/gc.c @@ -218,8 +218,8 @@ continue_scanning: key = rb_entry(cursor, struct key, serial_node); cursor = rb_next(cursor); - if (test_bit(KEY_FLAG_FINAL_PUT, &key->flags)) { - smp_mb(); /* Clobber key->user after FINAL_PUT seen. */ + if (!test_bit_acquire(KEY_FLAG_USER_ALIVE, &key->flags)) { + /* Clobber key->user after final put seen. */ goto found_unreferenced_key; } diff --git a/security/keys/key.c b/security/keys/key.c index 7198cd2ac3a3..3bbdde778631 100644 --- a/security/keys/key.c +++ b/security/keys/key.c @@ -298,6 +298,7 @@ struct key *key_alloc(struct key_type *type, const char *desc, key->restrict_link = restrict_link; key->last_used_at = ktime_get_real_seconds(); + key->flags |= 1 << KEY_FLAG_USER_ALIVE; if (!(flags & KEY_ALLOC_NOT_IN_QUOTA)) key->flags |= 1 << KEY_FLAG_IN_QUOTA; if (flags & KEY_ALLOC_BUILT_IN) @@ -658,8 +659,8 @@ void key_put(struct key *key) key->user->qnbytes -= key->quotalen; spin_unlock_irqrestore(&key->user->lock, flags); } - smp_mb(); /* key->user before FINAL_PUT set. */ - set_bit(KEY_FLAG_FINAL_PUT, &key->flags); + /* Mark key as safe for GC after key->user done. */ + clear_bit_unlock(KEY_FLAG_USER_ALIVE, &key->flags); schedule_work(&key_gc_work); } } diff --git a/sound/core/timer.c b/sound/core/timer.c index 1de4b90fd4d1..8072183c33d3 100644 --- a/sound/core/timer.c +++ b/sound/core/timer.c @@ -1118,8 +1118,8 @@ struct snd_timer_system_private { static void snd_timer_s_function(struct timer_list *t) { - struct snd_timer_system_private *priv = from_timer(priv, t, - tlist); + struct snd_timer_system_private *priv = timer_container_of(priv, t, + tlist); struct snd_timer *timer = priv->snd_timer; unsigned long jiff = jiffies; if (time_after(jiff, priv->last_expires)) diff --git a/sound/drivers/aloop.c b/sound/drivers/aloop.c index 4b02ec127cc0..6c318a5903ab 100644 --- a/sound/drivers/aloop.c +++ b/sound/drivers/aloop.c @@ -699,7 +699,7 @@ static unsigned int loopback_jiffies_timer_pos_update static void loopback_jiffies_timer_function(struct timer_list *t) { - struct loopback_pcm *dpcm = from_timer(dpcm, t, timer); + struct loopback_pcm *dpcm = timer_container_of(dpcm, t, timer); unsigned long flags; spin_lock_irqsave(&dpcm->cable->lock, flags); diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c index 1d923cbe8cd0..783fe3a22bc9 100644 --- a/sound/drivers/dummy.c +++ b/sound/drivers/dummy.c @@ -301,7 +301,7 @@ static int dummy_systimer_prepare(struct snd_pcm_substream *substream) static void dummy_systimer_callback(struct timer_list *t) { - struct dummy_systimer_pcm *dpcm = from_timer(dpcm, t, timer); + struct dummy_systimer_pcm *dpcm = timer_container_of(dpcm, t, timer); unsigned long flags; int elapsed = 0; diff --git a/sound/drivers/mpu401/mpu401_uart.c b/sound/drivers/mpu401/mpu401_uart.c index a63e7558ac07..670f8ba92c08 100644 --- a/sound/drivers/mpu401/mpu401_uart.c +++ b/sound/drivers/mpu401/mpu401_uart.c @@ -157,7 +157,7 @@ EXPORT_SYMBOL(snd_mpu401_uart_interrupt_tx); */ static void snd_mpu401_uart_timer(struct timer_list *t) { - struct snd_mpu401 *mpu = from_timer(mpu, t, timer); + struct snd_mpu401 *mpu = timer_container_of(mpu, t, timer); unsigned long flags; spin_lock_irqsave(&mpu->timer_lock, flags); diff --git a/sound/drivers/mtpav.c b/sound/drivers/mtpav.c index dffcdf9e10d4..851f34e2cdd0 100644 --- a/sound/drivers/mtpav.c +++ b/sound/drivers/mtpav.c @@ -388,7 +388,7 @@ static void snd_mtpav_input_trigger(struct snd_rawmidi_substream *substream, int static void snd_mtpav_output_timer(struct timer_list *t) { unsigned long flags; - struct mtpav *chip = from_timer(chip, t, timer); + struct mtpav *chip = timer_container_of(chip, t, timer); int p; spin_lock_irqsave(&chip->spinlock, flags); diff --git a/sound/drivers/opl3/opl3_midi.c b/sound/drivers/opl3/opl3_midi.c index 9bee454441b0..de7449cb6515 100644 --- a/sound/drivers/opl3/opl3_midi.c +++ b/sound/drivers/opl3/opl3_midi.c @@ -233,7 +233,7 @@ static int opl3_get_voice(struct snd_opl3 *opl3, int instr_4op, void snd_opl3_timer_func(struct timer_list *t) { - struct snd_opl3 *opl3 = from_timer(opl3, t, tlist); + struct snd_opl3 *opl3 = timer_container_of(opl3, t, tlist); unsigned long flags; int again = 0; int i; diff --git a/sound/drivers/pcmtest.c b/sound/drivers/pcmtest.c index 72378f354fd0..39f1e1fe4c44 100644 --- a/sound/drivers/pcmtest.c +++ b/sound/drivers/pcmtest.c @@ -345,7 +345,7 @@ static void timer_timeout(struct timer_list *data) struct pcmtst_buf_iter *v_iter; struct snd_pcm_substream *substream; - v_iter = from_timer(v_iter, data, timer_instance); + v_iter = timer_container_of(v_iter, data, timer_instance); substream = v_iter->substream; if (v_iter->suspend) diff --git a/sound/drivers/serial-u16550.c b/sound/drivers/serial-u16550.c index 1857a78f55d7..6d0656fcd574 100644 --- a/sound/drivers/serial-u16550.c +++ b/sound/drivers/serial-u16550.c @@ -299,7 +299,7 @@ static void snd_uart16550_buffer_timer(struct timer_list *t) unsigned long flags; struct snd_uart16550 *uart; - uart = from_timer(uart, t, buffer_timer); + uart = timer_container_of(uart, t, buffer_timer); spin_lock_irqsave(&uart->open_lock, flags); snd_uart16550_del_timer(uart); snd_uart16550_io_loop(uart); diff --git a/sound/i2c/other/ak4117.c b/sound/i2c/other/ak4117.c index 657b331d7a79..cd380db195ef 100644 --- a/sound/i2c/other/ak4117.c +++ b/sound/i2c/other/ak4117.c @@ -504,7 +504,7 @@ int snd_ak4117_check_rate_and_errors(struct ak4117 *ak4117, unsigned int flags) static void snd_ak4117_timer(struct timer_list *t) { - struct ak4117 *chip = from_timer(chip, t, timer); + struct ak4117 *chip = timer_container_of(chip, t, timer); if (chip->init) return; diff --git a/sound/isa/sb/emu8000_pcm.c b/sound/isa/sb/emu8000_pcm.c index 016235209928..215bbcd0360e 100644 --- a/sound/isa/sb/emu8000_pcm.c +++ b/sound/isa/sb/emu8000_pcm.c @@ -182,7 +182,7 @@ static inline int emu8k_get_curpos(struct snd_emu8k_pcm *rec, int ch) */ static void emu8k_pcm_timer_func(struct timer_list *t) { - struct snd_emu8k_pcm *rec = from_timer(rec, t, timer); + struct snd_emu8k_pcm *rec = timer_container_of(rec, t, timer); int ptr, delta; spin_lock(&rec->timer_lock); diff --git a/sound/isa/sb/sb8_midi.c b/sound/isa/sb/sb8_midi.c index 57867e51d367..637079a2f02a 100644 --- a/sound/isa/sb/sb8_midi.c +++ b/sound/isa/sb/sb8_midi.c @@ -200,7 +200,7 @@ static void snd_sb8dsp_midi_output_write(struct snd_rawmidi_substream *substream static void snd_sb8dsp_midi_output_timer(struct timer_list *t) { - struct snd_sb *chip = from_timer(chip, t, midi_timer); + struct snd_sb *chip = timer_container_of(chip, t, midi_timer); struct snd_rawmidi_substream *substream = chip->midi_substream_output; unsigned long flags; diff --git a/sound/isa/wavefront/wavefront_midi.c b/sound/isa/wavefront/wavefront_midi.c index fcc2a0d67792..494b21be665c 100644 --- a/sound/isa/wavefront/wavefront_midi.c +++ b/sound/isa/wavefront/wavefront_midi.c @@ -352,7 +352,7 @@ static void snd_wavefront_midi_input_trigger(struct snd_rawmidi_substream *subst static void snd_wavefront_midi_output_timer(struct timer_list *t) { - snd_wavefront_midi_t *midi = from_timer(midi, t, timer); + snd_wavefront_midi_t *midi = timer_container_of(midi, t, timer); snd_wavefront_card_t *card = midi->timer_card; unsigned long flags; diff --git a/sound/pci/asihpi/asihpi.c b/sound/pci/asihpi/asihpi.c index 7e5ce96954b9..cbd964f87349 100644 --- a/sound/pci/asihpi/asihpi.c +++ b/sound/pci/asihpi/asihpi.c @@ -709,7 +709,7 @@ static inline unsigned int modulo_min(unsigned int a, unsigned int b, */ static void snd_card_asihpi_timer_function(struct timer_list *t) { - struct snd_card_asihpi_pcm *dpcm = from_timer(dpcm, t, timer); + struct snd_card_asihpi_pcm *dpcm = timer_container_of(dpcm, t, timer); struct snd_pcm_substream *substream = dpcm->substream; struct snd_card_asihpi *card = snd_pcm_substream_chip(substream); struct snd_pcm_runtime *runtime; diff --git a/sound/pci/ctxfi/cttimer.c b/sound/pci/ctxfi/cttimer.c index aa179644b5c9..fa6867adb42b 100644 --- a/sound/pci/ctxfi/cttimer.c +++ b/sound/pci/ctxfi/cttimer.c @@ -62,7 +62,7 @@ struct ct_timer { static void ct_systimer_callback(struct timer_list *t) { - struct ct_timer_instance *ti = from_timer(ti, t, timer); + struct ct_timer_instance *ti = timer_container_of(ti, t, timer); struct snd_pcm_substream *substream = ti->substream; struct snd_pcm_runtime *runtime = substream->runtime; struct ct_atc_pcm *apcm = ti->apcm; diff --git a/sound/pci/echoaudio/midi.c b/sound/pci/echoaudio/midi.c index 2ef59184249c..c3f3c9129561 100644 --- a/sound/pci/echoaudio/midi.c +++ b/sound/pci/echoaudio/midi.c @@ -200,7 +200,7 @@ static int snd_echo_midi_output_open(struct snd_rawmidi_substream *substream) static void snd_echo_midi_output_write(struct timer_list *t) { - struct echoaudio *chip = from_timer(chip, t, timer); + struct echoaudio *chip = timer_container_of(chip, t, timer); unsigned long flags; int bytes, sent, time; unsigned char buf[MIDI_OUT_BUFFER_SIZE - 1]; diff --git a/sound/pci/hda/cs35l41_hda.c b/sound/pci/hda/cs35l41_hda.c index d9c8872b1866..d5bc81099d0d 100644 --- a/sound/pci/hda/cs35l41_hda.c +++ b/sound/pci/hda/cs35l41_hda.c @@ -1603,7 +1603,7 @@ static const struct regmap_irq cs35l41_reg_irqs[] = { CS35L41_REG_IRQ(IRQ1_STATUS1, AMP_SHORT_ERR), }; -static struct regmap_irq_chip cs35l41_regmap_irq_chip = { +static const struct regmap_irq_chip cs35l41_regmap_irq_chip = { .name = "cs35l41 IRQ1 Controller", .status_base = CS35L41_IRQ1_STATUS1, .mask_base = CS35L41_IRQ1_MASK1, diff --git a/sound/pci/hda/hda_bind.c b/sound/pci/hda/hda_bind.c index 1fef350d821e..df8f88beddd0 100644 --- a/sound/pci/hda/hda_bind.c +++ b/sound/pci/hda/hda_bind.c @@ -44,7 +44,7 @@ static void hda_codec_unsol_event(struct hdac_device *dev, unsigned int ev) struct hda_codec *codec = container_of(dev, struct hda_codec, core); /* ignore unsol events during shutdown */ - if (codec->bus->shutdown) + if (codec->card->shutdown || codec->bus->shutdown) return; /* ignore unsol events during system suspend/resume */ diff --git a/sound/pci/hda/hda_intel.c b/sound/pci/hda/hda_intel.c index e6df706f740d..e5210ed48ddf 100644 --- a/sound/pci/hda/hda_intel.c +++ b/sound/pci/hda/hda_intel.c @@ -2724,6 +2724,9 @@ static const struct pci_device_id azx_ids[] = { { PCI_VDEVICE(ATI, 0xab38), .driver_data = AZX_DRIVER_ATIHDMI_NS | AZX_DCAPS_PRESET_ATI_HDMI_NS | AZX_DCAPS_PM_RUNTIME }, + { PCI_VDEVICE(ATI, 0xab40), + .driver_data = AZX_DRIVER_ATIHDMI_NS | AZX_DCAPS_PRESET_ATI_HDMI_NS | + AZX_DCAPS_PM_RUNTIME }, /* GLENFLY */ { PCI_DEVICE(PCI_VENDOR_ID_GLENFLY, PCI_ANY_ID), .class = PCI_CLASS_MULTIMEDIA_HD_AUDIO << 8, diff --git a/sound/pci/rme9652/hdsp.c b/sound/pci/rme9652/hdsp.c index 873b7eadfc50..8df0f5bba0f6 100644 --- a/sound/pci/rme9652/hdsp.c +++ b/sound/pci/rme9652/hdsp.c @@ -1393,7 +1393,7 @@ static void snd_hdsp_midi_input_trigger(struct snd_rawmidi_substream *substream, static void snd_hdsp_midi_output_timer(struct timer_list *t) { - struct hdsp_midi *hmidi = from_timer(hmidi, t, timer); + struct hdsp_midi *hmidi = timer_container_of(hmidi, t, timer); unsigned long flags; snd_hdsp_midi_output_write(hmidi); diff --git a/sound/pci/rme9652/hdspm.c b/sound/pci/rme9652/hdspm.c index 64de54089955..2041cf00cca0 100644 --- a/sound/pci/rme9652/hdspm.c +++ b/sound/pci/rme9652/hdspm.c @@ -1943,7 +1943,7 @@ snd_hdspm_midi_input_trigger(struct snd_rawmidi_substream *substream, int up) static void snd_hdspm_midi_output_timer(struct timer_list *t) { - struct hdspm_midi *hmidi = from_timer(hmidi, t, timer); + struct hdspm_midi *hmidi = timer_container_of(hmidi, t, timer); unsigned long flags; snd_hdspm_midi_output_write(hmidi); diff --git a/sound/sh/aica.c b/sound/sh/aica.c index f88331a48638..40ea843113a7 100644 --- a/sound/sh/aica.c +++ b/sound/sh/aica.c @@ -284,8 +284,8 @@ static void run_spu_dma(struct work_struct *work) static void aica_period_elapsed(struct timer_list *t) { - struct snd_card_aica *dreamcastcard = from_timer(dreamcastcard, - t, timer); + struct snd_card_aica *dreamcastcard = timer_container_of(dreamcastcard, + t, timer); struct snd_pcm_substream *substream = dreamcastcard->substream; /*timer function - so cannot sleep */ int play_period; diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c index 3d9da93d22ee..7e62445e02c1 100644 --- a/sound/soc/amd/yc/acp6x-mach.c +++ b/sound/soc/amd/yc/acp6x-mach.c @@ -315,6 +315,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = { .driver_data = &acp6x_card, .matches = { DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"), + DMI_MATCH(DMI_PRODUCT_NAME, "83BS"), + } + }, + { + .driver_data = &acp6x_card, + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"), DMI_MATCH(DMI_PRODUCT_NAME, "83HN"), } }, diff --git a/sound/soc/codecs/es8375.c b/sound/soc/codecs/es8375.c index decc86c92427..009259632107 100644 --- a/sound/soc/codecs/es8375.c +++ b/sound/soc/codecs/es8375.c @@ -319,6 +319,7 @@ static int es8375_hw_params(struct snd_pcm_substream *substream, coeff = get_coeff(es8375->vddd, dmic_enable, es8375->mclk_freq, params_rate(params)); if (coeff < 0) { dev_warn(component->dev, "Clock coefficients do not match"); + return coeff; } regmap_write(es8375->regmap, ES8375_CLK_MGR4, coeff_div[coeff].Reg0x04); diff --git a/sound/soc/codecs/hda.c b/sound/soc/codecs/hda.c index ddc00927313c..dc7794c9ac44 100644 --- a/sound/soc/codecs/hda.c +++ b/sound/soc/codecs/hda.c @@ -152,7 +152,7 @@ int hda_codec_probe_complete(struct hda_codec *codec) ret = snd_hda_codec_build_controls(codec); if (ret < 0) { dev_err(&hdev->dev, "unable to create controls %d\n", ret); - goto out; + return ret; } /* Bus suspended codecs as it does not manage their pm */ @@ -160,7 +160,7 @@ int hda_codec_probe_complete(struct hda_codec *codec) /* rpm was forbidden in snd_hda_codec_device_new() */ snd_hda_codec_set_power_save(codec, 2000); snd_hda_codec_register(codec); -out: + /* Complement pm_runtime_get_sync(bus) in probe */ pm_runtime_mark_last_busy(bus->dev); pm_runtime_put_autosuspend(bus->dev); diff --git a/sound/soc/codecs/rt1320-sdw.c b/sound/soc/codecs/rt1320-sdw.c index f51ba345a16e..015cc710e6dc 100644 --- a/sound/soc/codecs/rt1320-sdw.c +++ b/sound/soc/codecs/rt1320-sdw.c @@ -204,7 +204,7 @@ static const struct reg_sequence rt1320_vc_blind_write[] = { { 0x3fc2bfc0, 0x03 }, { 0x0000d486, 0x43 }, { SDW_SDCA_CTL(FUNC_NUM_AMP, RT1320_SDCA_ENT_PDE23, RT1320_SDCA_CTL_REQ_POWER_STATE, 0), 0x00 }, - { 0x1000db00, 0x04 }, + { 0x1000db00, 0x07 }, { 0x1000db01, 0x00 }, { 0x1000db02, 0x11 }, { 0x1000db03, 0x00 }, @@ -225,6 +225,21 @@ static const struct reg_sequence rt1320_vc_blind_write[] = { { 0x1000db12, 0x00 }, { 0x1000db13, 0x00 }, { 0x1000db14, 0x45 }, + { 0x1000db15, 0x0d }, + { 0x1000db16, 0x01 }, + { 0x1000db17, 0x00 }, + { 0x1000db18, 0x00 }, + { 0x1000db19, 0xbf }, + { 0x1000db1a, 0x13 }, + { 0x1000db1b, 0x09 }, + { 0x1000db1c, 0x00 }, + { 0x1000db1d, 0x00 }, + { 0x1000db1e, 0x00 }, + { 0x1000db1f, 0x12 }, + { 0x1000db20, 0x09 }, + { 0x1000db21, 0x00 }, + { 0x1000db22, 0x00 }, + { 0x1000db23, 0x00 }, { 0x0000d540, 0x01 }, { 0x0000c081, 0xfc }, { 0x0000f01e, 0x80 }, diff --git a/sound/soc/codecs/rt5645.c b/sound/soc/codecs/rt5645.c index dba78efadc85..08df87238eee 100644 --- a/sound/soc/codecs/rt5645.c +++ b/sound/soc/codecs/rt5645.c @@ -3439,7 +3439,8 @@ static irqreturn_t rt5645_irq(int irq, void *data) static void rt5645_btn_check_callback(struct timer_list *t) { - struct rt5645_priv *rt5645 = from_timer(rt5645, t, btn_check_timer); + struct rt5645_priv *rt5645 = timer_container_of(rt5645, t, + btn_check_timer); queue_delayed_work(system_power_efficient_wq, &rt5645->jack_detect_work, msecs_to_jiffies(5)); diff --git a/sound/soc/codecs/tas571x.c b/sound/soc/codecs/tas571x.c index 6c6e7ae07d80..6bf37c77f0a7 100644 --- a/sound/soc/codecs/tas571x.c +++ b/sound/soc/codecs/tas571x.c @@ -718,6 +718,69 @@ static const struct regmap_config tas5721_regmap_config = { .volatile_table = &tas571x_volatile_regs, }; +static const struct snd_kcontrol_new tas5733_controls[] = { + /* MVOL LSB is ignored - see comments in tas571x_i2c_probe() */ + SOC_SINGLE_TLV("Master Volume", + TAS571X_MVOL_REG, 1, 0x1ff, 1, + tas5717_volume_tlv), + SOC_DOUBLE_R_TLV("Speaker Volume", + TAS571X_CH1_VOL_REG, TAS571X_CH2_VOL_REG, + 1, 0x1ff, 1, tas5717_volume_tlv), + SOC_DOUBLE("Speaker Switch", + TAS571X_SOFT_MUTE_REG, + TAS571X_SOFT_MUTE_CH1_SHIFT, TAS571X_SOFT_MUTE_CH2_SHIFT, + 1, 1), + + SOC_DOUBLE_R_RANGE("CH1 Mixer Volume", + TAS5717_CH1_LEFT_CH_MIX_REG, + TAS5717_CH1_RIGHT_CH_MIX_REG, + 16, 0, 0x80, 0), + + SOC_DOUBLE_R_RANGE("CH2 Mixer Volume", + TAS5717_CH2_LEFT_CH_MIX_REG, + TAS5717_CH2_RIGHT_CH_MIX_REG, + 16, 0, 0x80, 0), + + /* + * The biquads are named according to the register names. + * Please note that TI's TAS57xx Graphical Development Environment + * tool names them different. + */ + BIQUAD_COEFS("CH1 - Biquad 0", TAS5733_CH1_BQ0_REG), + BIQUAD_COEFS("CH1 - Biquad 1", TAS5733_CH1_BQ1_REG), + BIQUAD_COEFS("CH1 - Biquad 2", TAS5733_CH1_BQ2_REG), + BIQUAD_COEFS("CH1 - Biquad 3", TAS5733_CH1_BQ3_REG), + BIQUAD_COEFS("CH1 - Biquad 4", TAS5733_CH1_BQ4_REG), + BIQUAD_COEFS("CH1 - Biquad 5", TAS5733_CH1_BQ5_REG), + BIQUAD_COEFS("CH1 - Biquad 6", TAS5733_CH1_BQ6_REG), + BIQUAD_COEFS("CH1 - Biquad 7", TAS5733_CH1_BQ7_REG), + BIQUAD_COEFS("CH1 - Biquad 8", TAS5733_CH1_BQ8_REG), + BIQUAD_COEFS("CH1 - Biquad 9", TAS5733_CH1_BQ9_REG), + BIQUAD_COEFS("CH1 - Biquad 10", TAS5733_CH1_BQ10_REG), + + BIQUAD_COEFS("CH2 - Biquad 0", TAS5733_CH2_BQ0_REG), + BIQUAD_COEFS("CH2 - Biquad 1", TAS5733_CH2_BQ1_REG), + BIQUAD_COEFS("CH2 - Biquad 2", TAS5733_CH2_BQ2_REG), + BIQUAD_COEFS("CH2 - Biquad 3", TAS5733_CH2_BQ3_REG), + BIQUAD_COEFS("CH2 - Biquad 4", TAS5733_CH2_BQ4_REG), + BIQUAD_COEFS("CH2 - Biquad 5", TAS5733_CH2_BQ5_REG), + BIQUAD_COEFS("CH2 - Biquad 6", TAS5733_CH2_BQ6_REG), + BIQUAD_COEFS("CH2 - Biquad 7", TAS5733_CH2_BQ7_REG), + BIQUAD_COEFS("CH2 - Biquad 8", TAS5733_CH2_BQ8_REG), + BIQUAD_COEFS("CH2 - Biquad 9", TAS5733_CH2_BQ9_REG), + BIQUAD_COEFS("CH2 - Biquad 10", TAS5733_CH2_BQ10_REG), + + BIQUAD_COEFS("CH1 - Cross Biquad 0", TAS5733_CH1_CBQ0_REG), + BIQUAD_COEFS("CH1 - Cross Biquad 1", TAS5733_CH1_CBQ1_REG), + BIQUAD_COEFS("CH1 - Cross Biquad 2", TAS5733_CH1_CBQ2_REG), + BIQUAD_COEFS("CH1 - Cross Biquad 3", TAS5733_CH1_CBQ3_REG), + + BIQUAD_COEFS("CH2 - Cross Biquad 0", TAS5733_CH2_CBQ0_REG), + BIQUAD_COEFS("CH2 - Cross Biquad 1", TAS5733_CH2_CBQ1_REG), + BIQUAD_COEFS("CH2 - Cross Biquad 2", TAS5733_CH2_CBQ2_REG), + BIQUAD_COEFS("CH2 - Cross Biquad 3", TAS5733_CH2_CBQ3_REG), +}; + static const char *const tas5733_supply_names[] = { "AVDD", "DVDD", @@ -770,8 +833,8 @@ static const struct regmap_config tas5733_regmap_config = { static const struct tas571x_chip tas5733_chip = { .supply_names = tas5733_supply_names, .num_supply_names = ARRAY_SIZE(tas5733_supply_names), - .controls = tas5717_controls, - .num_controls = ARRAY_SIZE(tas5717_controls), + .controls = tas5733_controls, + .num_controls = ARRAY_SIZE(tas5733_controls), .regmap_config = &tas5733_regmap_config, .vol_reg_size = 2, }; diff --git a/sound/soc/codecs/tas571x.h b/sound/soc/codecs/tas571x.h index 5340d3bec31d..2b3eff4023b9 100644 --- a/sound/soc/codecs/tas571x.h +++ b/sound/soc/codecs/tas571x.h @@ -104,4 +104,38 @@ #define TAS5717_CH2_LEFT_CH_MIX_REG 0x76 #define TAS5717_CH2_RIGHT_CH_MIX_REG 0x77 +#define TAS5733_CH1_BQ0_REG 0x26 +#define TAS5733_CH1_BQ1_REG 0x27 +#define TAS5733_CH1_BQ2_REG 0x28 +#define TAS5733_CH1_BQ3_REG 0x29 +#define TAS5733_CH1_BQ4_REG 0x2a +#define TAS5733_CH1_BQ5_REG 0x2b +#define TAS5733_CH1_BQ6_REG 0x2c +#define TAS5733_CH1_BQ7_REG 0x2d +#define TAS5733_CH1_BQ8_REG 0x2e +#define TAS5733_CH1_BQ9_REG 0x2f + +#define TAS5733_CH2_BQ0_REG 0x30 +#define TAS5733_CH2_BQ1_REG 0x31 +#define TAS5733_CH2_BQ2_REG 0x32 +#define TAS5733_CH2_BQ3_REG 0x33 +#define TAS5733_CH2_BQ4_REG 0x34 +#define TAS5733_CH2_BQ5_REG 0x35 +#define TAS5733_CH2_BQ6_REG 0x36 +#define TAS5733_CH2_BQ7_REG 0x37 +#define TAS5733_CH2_BQ8_REG 0x38 +#define TAS5733_CH2_BQ9_REG 0x39 + +#define TAS5733_CH1_BQ10_REG 0x58 +#define TAS5733_CH1_CBQ0_REG 0x59 +#define TAS5733_CH1_CBQ1_REG 0x5a +#define TAS5733_CH1_CBQ2_REG 0x5b +#define TAS5733_CH1_CBQ3_REG 0x5c + +#define TAS5733_CH2_BQ10_REG 0x5d +#define TAS5733_CH2_CBQ0_REG 0x5e +#define TAS5733_CH2_CBQ1_REG 0x5f +#define TAS5733_CH2_CBQ2_REG 0x60 +#define TAS5733_CH2_CBQ3_REG 0x61 + #endif /* _TAS571X_H */ diff --git a/sound/soc/codecs/wcd9335.c b/sound/soc/codecs/wcd9335.c index 8ee4360aff92..5e19e813748d 100644 --- a/sound/soc/codecs/wcd9335.c +++ b/sound/soc/codecs/wcd9335.c @@ -332,7 +332,6 @@ struct wcd9335_codec { int intr1; struct gpio_desc *reset_gpio; - struct regulator_bulk_data supplies[WCD9335_MAX_SUPPLY]; unsigned int rx_port_value[WCD9335_RX_MAX]; unsigned int tx_port_value[WCD9335_TX_MAX]; @@ -355,6 +354,10 @@ struct wcd9335_irq { char *name; }; +static const char * const wcd9335_supplies[] = { + "vdd-buck", "vdd-buck-sido", "vdd-tx", "vdd-rx", "vdd-io", +}; + static const struct wcd9335_slim_ch wcd9335_tx_chs[WCD9335_TX_MAX] = { WCD9335_SLIM_TX_CH(0), WCD9335_SLIM_TX_CH(1), @@ -4989,30 +4992,16 @@ static int wcd9335_parse_dt(struct wcd9335_codec *wcd) if (IS_ERR(wcd->native_clk)) return dev_err_probe(dev, PTR_ERR(wcd->native_clk), "slimbus clock not found\n"); - wcd->supplies[0].supply = "vdd-buck"; - wcd->supplies[1].supply = "vdd-buck-sido"; - wcd->supplies[2].supply = "vdd-tx"; - wcd->supplies[3].supply = "vdd-rx"; - wcd->supplies[4].supply = "vdd-io"; - - ret = regulator_bulk_get(dev, WCD9335_MAX_SUPPLY, wcd->supplies); + ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(wcd9335_supplies), + wcd9335_supplies); if (ret) - return dev_err_probe(dev, ret, "Failed to get supplies\n"); + return dev_err_probe(dev, ret, "Failed to get and enable supplies\n"); return 0; } static int wcd9335_power_on_reset(struct wcd9335_codec *wcd) { - struct device *dev = wcd->dev; - int ret; - - ret = regulator_bulk_enable(WCD9335_MAX_SUPPLY, wcd->supplies); - if (ret) { - dev_err(dev, "Failed to get supplies: err = %d\n", ret); - return ret; - } - /* * For WCD9335, it takes about 600us for the Vout_A and * Vout_D to be ready after BUCK_SIDO is powered up. diff --git a/sound/soc/codecs/wcd937x.c b/sound/soc/codecs/wcd937x.c index 3b1a1518e764..b9df58b86ce9 100644 --- a/sound/soc/codecs/wcd937x.c +++ b/sound/soc/codecs/wcd937x.c @@ -91,7 +91,6 @@ struct wcd937x_priv { struct regmap_irq_chip *wcd_regmap_irq_chip; struct regmap_irq_chip_data *irq_chip; struct regulator_bulk_data supplies[WCD937X_MAX_BULK_SUPPLY]; - struct regulator *buck_supply; struct snd_soc_jack *jack; unsigned long status_mask; s32 micb_ref[WCD937X_MAX_MICBIAS]; @@ -2945,10 +2944,8 @@ static int wcd937x_probe(struct platform_device *pdev) return dev_err_probe(dev, ret, "Failed to get supplies\n"); ret = regulator_bulk_enable(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); - if (ret) { - regulator_bulk_free(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); + if (ret) return dev_err_probe(dev, ret, "Failed to enable supplies\n"); - } wcd937x_dt_parse_micbias_info(dev, wcd937x); @@ -2984,7 +2981,6 @@ static int wcd937x_probe(struct platform_device *pdev) err_disable_regulators: regulator_bulk_disable(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); - regulator_bulk_free(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); return ret; } @@ -3001,7 +2997,6 @@ static void wcd937x_remove(struct platform_device *pdev) pm_runtime_dont_use_autosuspend(dev); regulator_bulk_disable(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); - regulator_bulk_free(WCD937X_MAX_BULK_SUPPLY, wcd937x->supplies); } #if defined(CONFIG_OF) diff --git a/sound/soc/fsl/imx-pcm-rpmsg.c b/sound/soc/fsl/imx-pcm-rpmsg.c index 8ed62d43ffd5..edab68ae8366 100644 --- a/sound/soc/fsl/imx-pcm-rpmsg.c +++ b/sound/soc/fsl/imx-pcm-rpmsg.c @@ -209,7 +209,7 @@ static snd_pcm_uframes_t imx_rpmsg_pcm_pointer(struct snd_soc_component *compone static void imx_rpmsg_timer_callback(struct timer_list *t) { struct stream_timer *stream_timer = - from_timer(stream_timer, t, timer); + timer_container_of(stream_timer, t, timer); struct snd_pcm_substream *substream = stream_timer->substream; struct rpmsg_info *info = stream_timer->info; struct rpmsg_msg *msg; diff --git a/sound/soc/intel/avs/board_selection.c b/sound/soc/intel/avs/board_selection.c index 636315060eb4..673ccf162023 100644 --- a/sound/soc/intel/avs/board_selection.c +++ b/sound/soc/intel/avs/board_selection.c @@ -548,7 +548,7 @@ static int avs_register_i2s_test_boards(struct avs_dev *adev) u32 *array, num_elems; ret = parse_int_array(i2s_test, strlen(i2s_test), (int **)&array); - if (ret < 0) { + if (ret) { dev_err(adev->dev, "failed to parse i2s_test parameter\n"); return ret; } diff --git a/sound/soc/intel/avs/boards/rt5663.c b/sound/soc/intel/avs/boards/rt5663.c index 122b6c48fd80..51648801710a 100644 --- a/sound/soc/intel/avs/boards/rt5663.c +++ b/sound/soc/intel/avs/boards/rt5663.c @@ -228,7 +228,7 @@ static int avs_rt5663_probe(struct platform_device *pdev) card->name = "avs_rt5663"; } else { card->driver_name = "avs_rt5663"; - card->long_name = card->name = "AVS I2S ALC5640"; + card->long_name = card->name = "AVS I2S ALC5663"; } card->dev = dev; card->owner = THIS_MODULE; diff --git a/sound/soc/intel/avs/core.c b/sound/soc/intel/avs/core.c index 485842838025..ec1b3f55cb5c 100644 --- a/sound/soc/intel/avs/core.c +++ b/sound/soc/intel/avs/core.c @@ -945,14 +945,14 @@ MODULE_AUTHOR("Cezary Rojewski <cezary.rojewski@intel.com>"); MODULE_AUTHOR("Amadeusz Slawinski <amadeuszx.slawinski@linux.intel.com>"); MODULE_DESCRIPTION("Intel cAVS sound driver"); MODULE_LICENSE("GPL"); -MODULE_FIRMWARE("intel/skl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/apl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/cnl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/icl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/jsl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/lkf/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/tgl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/ehl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/adl/dsp_basefw.bin"); -MODULE_FIRMWARE("intel/adl_n/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/skl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/apl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/cnl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/icl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/jsl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/lkf/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/tgl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/ehl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/adl/dsp_basefw.bin"); +MODULE_FIRMWARE("intel/avs/adl_n/dsp_basefw.bin"); MODULE_FIRMWARE("intel/fcl/dsp_basefw.bin"); diff --git a/sound/soc/intel/avs/debugfs.c b/sound/soc/intel/avs/debugfs.c index 8c4edda97f75..c625cf879f17 100644 --- a/sound/soc/intel/avs/debugfs.c +++ b/sound/soc/intel/avs/debugfs.c @@ -144,7 +144,7 @@ static ssize_t probe_points_write(struct file *file, const char __user *from, si int ret; ret = parse_int_array_user(from, count, (int **)&array); - if (ret < 0) + if (ret) return ret; num_elems = *array; @@ -181,7 +181,7 @@ static ssize_t probe_points_disconnect_write(struct file *file, const char __use int ret; ret = parse_int_array_user(from, count, (int **)&array); - if (ret < 0) + if (ret) return ret; num_elems = *array; @@ -369,11 +369,14 @@ static ssize_t trace_control_write(struct file *file, const char __user *from, s int ret; ret = parse_int_array_user(from, count, (int **)&array); - if (ret < 0) + if (ret) return ret; num_elems = *array; - resource_mask = array[1]; + if (!num_elems) { + ret = -EINVAL; + goto free_array; + } /* * Disable if just resource mask is provided - no log priority flags. @@ -381,6 +384,7 @@ static ssize_t trace_control_write(struct file *file, const char __user *from, s * Enable input format: mask, prio1, .., prioN * Where 'N' equals number of bits set in the 'mask'. */ + resource_mask = array[1]; if (num_elems == 1) { ret = disable_logs(adev, resource_mask); } else { diff --git a/sound/soc/intel/avs/ipc.c b/sound/soc/intel/avs/ipc.c index 08ed9d96738a..0314f9d4ea5f 100644 --- a/sound/soc/intel/avs/ipc.c +++ b/sound/soc/intel/avs/ipc.c @@ -169,7 +169,9 @@ static void avs_dsp_exception_caught(struct avs_dev *adev, union avs_notify_msg dev_crit(adev->dev, "communication severed, rebooting dsp..\n"); - cancel_delayed_work_sync(&ipc->d0ix_work); + /* Avoid deadlock as the exception may be the response to SET_D0IX. */ + if (current_work() != &ipc->d0ix_work.work) + cancel_delayed_work_sync(&ipc->d0ix_work); ipc->in_d0ix = false; /* Re-enabled on recovery completion. */ pm_runtime_disable(adev->dev); diff --git a/sound/soc/intel/avs/loader.c b/sound/soc/intel/avs/loader.c index 138e4e9de5e3..353e343b1d28 100644 --- a/sound/soc/intel/avs/loader.c +++ b/sound/soc/intel/avs/loader.c @@ -9,6 +9,7 @@ #include <linux/firmware.h> #include <linux/module.h> #include <linux/slab.h> +#include <linux/string.h> #include <sound/hdaudio.h> #include <sound/hdaudio_ext.h> #include "avs.h" diff --git a/sound/soc/intel/avs/path.c b/sound/soc/intel/avs/path.c index ed8f0ea0e10d..e8e6b1c7fc90 100644 --- a/sound/soc/intel/avs/path.c +++ b/sound/soc/intel/avs/path.c @@ -134,6 +134,8 @@ int avs_path_set_constraint(struct avs_dev *adev, struct avs_tplg_path_template rlist = kcalloc(i, sizeof(*rlist), GFP_KERNEL); clist = kcalloc(i, sizeof(*clist), GFP_KERNEL); slist = kcalloc(i, sizeof(*slist), GFP_KERNEL); + if (!rlist || !clist || !slist) + return -ENOMEM; i = 0; list_for_each_entry(path_template, &template->path_list, node) { diff --git a/sound/soc/intel/avs/pcm.c b/sound/soc/intel/avs/pcm.c index 405cfc1ab0cb..ccf90428126d 100644 --- a/sound/soc/intel/avs/pcm.c +++ b/sound/soc/intel/avs/pcm.c @@ -83,10 +83,8 @@ void avs_period_elapsed(struct snd_pcm_substream *substream) static int hw_rule_param_size(struct snd_pcm_hw_params *params, struct snd_pcm_hw_rule *rule); static int avs_hw_constraints_init(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) { - struct snd_soc_pcm_runtime *rtd = snd_soc_substream_to_rtd(substream); struct snd_pcm_runtime *runtime = substream->runtime; struct snd_pcm_hw_constraint_list *r, *c, *s; - struct avs_tplg_path_template *template; struct avs_dma_data *data; int ret; @@ -99,8 +97,7 @@ static int avs_hw_constraints_init(struct snd_pcm_substream *substream, struct s c = &(data->channels_list); s = &(data->sample_bits_list); - template = avs_dai_find_path_template(dai, !rtd->dai_link->no_pcm, substream->stream); - ret = avs_path_set_constraint(data->adev, template, r, c, s); + ret = avs_path_set_constraint(data->adev, data->template, r, c, s); if (ret <= 0) return ret; @@ -450,9 +447,10 @@ static int avs_dai_hda_be_hw_free(struct snd_pcm_substream *substream, struct sn static int avs_dai_hda_be_prepare(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) { - struct snd_pcm_runtime *runtime = substream->runtime; + struct snd_soc_pcm_runtime *be = snd_soc_substream_to_rtd(substream); const struct snd_soc_pcm_stream *stream_info; struct hdac_ext_stream *link_stream; + const struct snd_pcm_hw_params *p; struct avs_dma_data *data; unsigned int format_val; unsigned int bits; @@ -460,14 +458,15 @@ static int avs_dai_hda_be_prepare(struct snd_pcm_substream *substream, struct sn data = snd_soc_dai_get_dma_data(dai, substream); link_stream = data->link_stream; + p = &be->dpcm[substream->stream].hw_params; if (link_stream->link_prepared) return 0; stream_info = snd_soc_dai_get_pcm_stream(dai, substream->stream); - bits = snd_hdac_stream_format_bits(runtime->format, runtime->subformat, + bits = snd_hdac_stream_format_bits(params_format(p), params_subformat(p), stream_info->sig_bits); - format_val = snd_hdac_stream_format(runtime->channels, bits, runtime->rate); + format_val = snd_hdac_stream_format(params_channels(p), bits, params_rate(p)); snd_hdac_ext_stream_decouple(&data->adev->base.core, link_stream, true); snd_hdac_ext_stream_reset(link_stream); diff --git a/sound/soc/soc-pcm.c b/sound/soc/soc-pcm.c index 43835197d1fe..2c21fd528afd 100644 --- a/sound/soc/soc-pcm.c +++ b/sound/soc/soc-pcm.c @@ -2510,17 +2510,6 @@ static int dpcm_fe_dai_prepare(struct snd_pcm_substream *substream) dpcm_set_fe_update_state(fe, stream, SND_SOC_DPCM_UPDATE_FE); - /* there is no point preparing this FE if there are no BEs */ - if (list_empty(&fe->dpcm[stream].be_clients)) { - /* dev_err_once() for visibility, dev_dbg() for debugging UCM profiles */ - dev_err_once(fe->dev, "ASoC: no backend DAIs enabled for %s, possibly missing ALSA mixer-based routing or UCM profile\n", - fe->dai_link->name); - dev_dbg(fe->dev, "ASoC: no backend DAIs enabled for %s\n", - fe->dai_link->name); - ret = -EINVAL; - goto out; - } - ret = dpcm_be_dai_prepare(fe, stream); if (ret < 0) goto out; @@ -2776,11 +2765,23 @@ static int dpcm_fe_dai_open(struct snd_pcm_substream *fe_substream) /* calculate valid and active FE <-> BE dpcms */ dpcm_add_paths(fe, stream, &list); + /* There is no point starting up this FE if there are no BEs. */ + if (list_empty(&fe->dpcm[stream].be_clients)) { + /* dev_err_once() for visibility, dev_dbg() for debugging UCM profiles. */ + dev_err_once(fe->dev, "ASoC: no backend DAIs enabled for %s, possibly missing ALSA mixer-based routing or UCM profile\n", + fe->dai_link->name); + dev_dbg(fe->dev, "ASoC: no backend DAIs enabled for %s\n", fe->dai_link->name); + + ret = -EINVAL; + goto put_path; + } + ret = dpcm_fe_dai_startup(fe_substream); if (ret < 0) dpcm_fe_dai_cleanup(fe_substream); dpcm_clear_pending_state(fe, stream); +put_path: dpcm_path_put(&list); open_end: snd_soc_dpcm_mutex_unlock(fe); diff --git a/sound/soc/ti/omap-hdmi.c b/sound/soc/ti/omap-hdmi.c index cf43ac19c4a6..55e7cb96858f 100644 --- a/sound/soc/ti/omap-hdmi.c +++ b/sound/soc/ti/omap-hdmi.c @@ -361,17 +361,20 @@ static int omap_hdmi_audio_probe(struct platform_device *pdev) if (!card->dai_link) return -ENOMEM; - compnent = devm_kzalloc(dev, sizeof(*compnent), GFP_KERNEL); + compnent = devm_kzalloc(dev, 2 * sizeof(*compnent), GFP_KERNEL); if (!compnent) return -ENOMEM; - card->dai_link->cpus = compnent; + card->dai_link->cpus = &compnent[0]; card->dai_link->num_cpus = 1; card->dai_link->codecs = &snd_soc_dummy_dlc; card->dai_link->num_codecs = 1; + card->dai_link->platforms = &compnent[1]; + card->dai_link->num_platforms = 1; card->dai_link->name = card->name; card->dai_link->stream_name = card->name; card->dai_link->cpus->dai_name = dev_name(ad->dssdev); + card->dai_link->platforms->name = dev_name(ad->dssdev); card->num_links = 1; card->dev = dev; diff --git a/sound/synth/emux/emux_synth.c b/sound/synth/emux/emux_synth.c index 02e2c69d7f18..cff6aba9bfc3 100644 --- a/sound/synth/emux/emux_synth.c +++ b/sound/synth/emux/emux_synth.c @@ -190,7 +190,7 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan) */ void snd_emux_timer_callback(struct timer_list *t) { - struct snd_emux *emu = from_timer(emu, t, tlist); + struct snd_emux *emu = timer_container_of(emu, t, tlist); struct snd_emux_voice *vp; unsigned long flags; int ch, do_again = 0; diff --git a/sound/usb/implicit.c b/sound/usb/implicit.c index 4727043fd745..77f06da93151 100644 --- a/sound/usb/implicit.c +++ b/sound/usb/implicit.c @@ -57,6 +57,7 @@ static const struct snd_usb_implicit_fb_match playback_implicit_fb_quirks[] = { IMPLICIT_FB_FIXED_DEV(0x31e9, 0x0002, 0x81, 2), /* Solid State Logic SSL2+ */ IMPLICIT_FB_FIXED_DEV(0x0499, 0x172f, 0x81, 2), /* Steinberg UR22C */ IMPLICIT_FB_FIXED_DEV(0x0d9a, 0x00df, 0x81, 2), /* RTX6001 */ + IMPLICIT_FB_FIXED_DEV(0x19f7, 0x000a, 0x84, 3), /* RODE AI-1 */ IMPLICIT_FB_FIXED_DEV(0x22f0, 0x0006, 0x81, 3), /* Allen&Heath Qu-16 */ IMPLICIT_FB_FIXED_DEV(0x1686, 0xf029, 0x82, 2), /* Zoom UAC-2 */ IMPLICIT_FB_FIXED_DEV(0x2466, 0x8003, 0x86, 2), /* Fractal Audio Axe-Fx II */ diff --git a/sound/usb/midi.c b/sound/usb/midi.c index 3a8a977ed359..866e613fee4f 100644 --- a/sound/usb/midi.c +++ b/sound/usb/midi.c @@ -339,7 +339,7 @@ static void snd_usbmidi_out_work(struct work_struct *work) /* called after transfers had been interrupted due to some USB error */ static void snd_usbmidi_error_timer(struct timer_list *t) { - struct snd_usb_midi *umidi = from_timer(umidi, t, error_timer); + struct snd_usb_midi *umidi = timer_container_of(umidi, t, error_timer); unsigned int i, j; spin_lock(&umidi->disc_lock); diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c index 5ed523b13fad..bd24f3a78ea9 100644 --- a/sound/usb/quirks.c +++ b/sound/usb/quirks.c @@ -2285,6 +2285,8 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { QUIRK_FLAG_DISABLE_AUTOSUSPEND), DEVICE_FLG(0x17aa, 0x104d, /* Lenovo ThinkStation P620 Internal Speaker + Front Headset */ QUIRK_FLAG_DISABLE_AUTOSUSPEND), + DEVICE_FLG(0x17ef, 0x3083, /* Lenovo TBT3 dock */ + QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x1852, 0x5062, /* Luxman D-08u */ QUIRK_FLAG_ITF_USB_DSD_DAC | QUIRK_FLAG_CTL_MSG_DELAY), DEVICE_FLG(0x1852, 0x5065, /* Luxman DA-06 */ diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c index 01818abd02df..6d2c280a1730 100644 --- a/tools/perf/util/symbol-elf.c +++ b/tools/perf/util/symbol-elf.c @@ -1668,6 +1668,12 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss, continue; } + /* Reject RISCV ELF "mapping symbols" */ + if (ehdr.e_machine == EM_RISCV) { + if (elf_name[0] == '$' && strchr("dx", elf_name[1])) + continue; + } + if (runtime_ss->opdsec && sym.st_shndx == runtime_ss->opdidx) { u32 offset = sym.st_value - syms_ss->opdshdr.sh_addr; u64 *opd = opddata->d_buf + offset; diff --git a/tools/power/cpupower/Makefile b/tools/power/cpupower/Makefile index be8dfac14076..c43db1c41205 100644 --- a/tools/power/cpupower/Makefile +++ b/tools/power/cpupower/Makefile @@ -73,6 +73,7 @@ sbindir ?= /usr/sbin mandir ?= /usr/man libdir ?= /usr/lib libexecdir ?= /usr/libexec +unitdir ?= /usr/lib/systemd/system includedir ?= /usr/include localedir ?= /usr/share/locale docdir ?= /usr/share/doc/packages/cpupower @@ -309,9 +310,9 @@ install-tools: $(OUTPUT)cpupower $(INSTALL_DATA) cpupower-service.conf '$(DESTDIR)${confdir}' $(INSTALL) -d $(DESTDIR)${libexecdir} $(INSTALL_PROGRAM) cpupower.sh '$(DESTDIR)${libexecdir}/cpupower' - $(INSTALL) -d $(DESTDIR)${libdir}/systemd/system - sed 's|___CDIR___|${confdir}|; s|___LDIR___|${libexecdir}|' cpupower.service.in > '$(DESTDIR)${libdir}/systemd/system/cpupower.service' - $(SETPERM_DATA) '$(DESTDIR)${libdir}/systemd/system/cpupower.service' + $(INSTALL) -d $(DESTDIR)${unitdir} + sed 's|___CDIR___|${confdir}|; s|___LDIR___|${libexecdir}|' cpupower.service.in > '$(DESTDIR)${unitdir}/cpupower.service' + $(SETPERM_DATA) '$(DESTDIR)${unitdir}/cpupower.service' install-man: $(INSTALL_DATA) -D man/cpupower.1 $(DESTDIR)${mandir}/man1/cpupower.1 @@ -348,7 +349,7 @@ uninstall: - rm -f $(DESTDIR)${bindir}/utils/cpupower - rm -f $(DESTDIR)${confdir}cpupower-service.conf - rm -f $(DESTDIR)${libexecdir}/cpupower - - rm -f $(DESTDIR)${libdir}/systemd/system/cpupower.service + - rm -f $(DESTDIR)${unitdir}/cpupower.service - rm -f $(DESTDIR)${mandir}/man1/cpupower.1 - rm -f $(DESTDIR)${mandir}/man1/cpupower-frequency-set.1 - rm -f $(DESTDIR)${mandir}/man1/cpupower-frequency-info.1 diff --git a/tools/power/x86/turbostat/turbostat.8 b/tools/power/x86/turbostat/turbostat.8 index b74ed916057e..fb11108aaf42 100644 --- a/tools/power/x86/turbostat/turbostat.8 +++ b/tools/power/x86/turbostat/turbostat.8 @@ -100,7 +100,7 @@ The column name "all" can be used to enable all disabled-by-default built-in cou .PP \fB--show column\fP show only the specified built-in columns. May be invoked multiple times, or with a comma-separated list of column names. .PP -\fB--show CATEGORY --hide CATEGORY\fP Show and hide also accept a single CATEGORY of columns: "all", "topology", "idle", "frequency", "power", "cpuidle", "hwidle", "swidle", "other". "idle" (enabled by default), includes "hwidle" and "idle_pct". "cpuidle" (default disabled) includes cpuidle software invocation counters. "swidle" includes "cpuidle" plus "idle_pct". "hwidle" includes only hardware based idle residency counters. Older versions of turbostat used the term "sysfs" for what is now "swidle". +\fB--show CATEGORY --hide CATEGORY\fP Show and hide also accept a single CATEGORY of columns: "all", "topology", "idle", "frequency", "power", "cpuidle", "hwidle", "swidle", "other". "idle" (enabled by default), includes "hwidle" and "pct_idle". "cpuidle" (default disabled) includes cpuidle software invocation counters. "swidle" includes "cpuidle" plus "pct_idle". "hwidle" includes only hardware based idle residency counters. Older versions of turbostat used the term "sysfs" for what is now "swidle". .PP \fB--Dump\fP displays the raw counter values. .PP @@ -204,8 +204,8 @@ The system configuration dump (if --quiet is not used) is followed by statistics .PP \fBUncMHz\fP per-package uncore MHz, instantaneous sample. .PP -\fBUMHz1.0\fP per-package uncore MHz for domain=1 and fabric_cluster=0, instantaneous sample. System summary is the average of all packages. -Intel Granite Rapids systems use domains 0-2 for CPUs, and 3-4 for IO, with cluster always 0. +\fBUMHz1.0\fP per-package uncore MHz for pm_domain=1 and fabric_cluster=0, instantaneous sample. System summary is the average of all packages. +Intel Granite Rapids systems use pm_domains 0-2 for CPUs, and 3-4 for IO, with cluster always 0. For the "--show" and "--hide" options, use "UncMHz" to operate on all UMHz*.* as a group. .SH TOO MUCH INFORMATION EXAMPLE By default, turbostat dumps all possible information -- a system configuration header, followed by columns for all counters. diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbostat/turbostat.c index 0170d3cc6819..5230e072e414 100644 --- a/tools/power/x86/turbostat/turbostat.c +++ b/tools/power/x86/turbostat/turbostat.c @@ -280,7 +280,7 @@ struct msr_counter bic[] = { #define BIC_GROUP_FREQUENCY (BIC_Avg_MHz | BIC_Busy | BIC_Bzy_MHz | BIC_TSC_MHz | BIC_GFXMHz | BIC_GFXACTMHz | BIC_SAMMHz | BIC_SAMACTMHz | BIC_UNCORE_MHZ) #define BIC_GROUP_HW_IDLE (BIC_Busy | BIC_CPU_c1 | BIC_CPU_c3 | BIC_CPU_c6 | BIC_CPU_c7 | BIC_GFX_rc6 | BIC_Pkgpc2 | BIC_Pkgpc3 | BIC_Pkgpc6 | BIC_Pkgpc7 | BIC_Pkgpc8 | BIC_Pkgpc9 | BIC_Pkgpc10 | BIC_CPU_LPI | BIC_SYS_LPI | BIC_Mod_c6 | BIC_Totl_c0 | BIC_Any_c0 | BIC_GFX_c0 | BIC_CPUGFX | BIC_SAM_mc6 | BIC_Diec6) #define BIC_GROUP_SW_IDLE (BIC_Busy | BIC_cpuidle | BIC_pct_idle ) -#define BIC_GROUP_IDLE (BIC_GROUP_HW_IDLE | BIC_pct_idle) +#define BIC_GROUP_IDLE (BIC_GROUP_HW_IDLE | BIC_pct_idle) #define BIC_OTHER (BIC_IRQ | BIC_NMI | BIC_SMI | BIC_ThreadC | BIC_CoreTmp | BIC_IPC) #define BIC_DISABLED_BY_DEFAULT (BIC_USEC | BIC_TOD | BIC_APIC | BIC_X2APIC | BIC_cpuidle) @@ -539,7 +539,7 @@ enum rapl_msrs { #define RAPL_PKG_ALL (RAPL_PKG | RAPL_PKG_PERF_STATUS | RAPL_PKG_POWER_INFO) #define RAPL_DRAM_ALL (RAPL_DRAM | RAPL_DRAM_PERF_STATUS | RAPL_DRAM_POWER_INFO) #define RAPL_CORE_ALL (RAPL_CORE | RAPL_CORE_POLICY) -#define RAPL_GFX_ALL (RAPL_GFX | RAPL_GFX_POLIGY) +#define RAPL_GFX_ALL (RAPL_GFX | RAPL_GFX_POLICY) #define RAPL_AMD_F17H (RAPL_AMD_PWR_UNIT | RAPL_AMD_CORE_ENERGY_STAT | RAPL_AMD_PKG_ENERGY_STAT) @@ -839,6 +839,23 @@ static const struct platform_features spr_features = { .rapl_msrs = RAPL_PKG_ALL | RAPL_DRAM_ALL | RAPL_PSYS, }; +static const struct platform_features dmr_features = { + .has_msr_misc_feature_control = spr_features.has_msr_misc_feature_control, + .has_msr_misc_pwr_mgmt = spr_features.has_msr_misc_pwr_mgmt, + .has_nhm_msrs = spr_features.has_nhm_msrs, + .has_config_tdp = spr_features.has_config_tdp, + .bclk_freq = spr_features.bclk_freq, + .supported_cstates = spr_features.supported_cstates, + .cst_limit = spr_features.cst_limit, + .has_msr_core_c1_res = spr_features.has_msr_core_c1_res, + .has_msr_module_c6_res_ms = 1, /* DMR has Dual Core Module and MC6 MSR */ + .has_irtl_msrs = spr_features.has_irtl_msrs, + .has_cst_prewake_bit = spr_features.has_cst_prewake_bit, + .has_fixed_rapl_psys_unit = spr_features.has_fixed_rapl_psys_unit, + .trl_msrs = spr_features.trl_msrs, + .rapl_msrs = 0, /* DMR does not have RAPL MSRs */ +}; + static const struct platform_features srf_features = { .has_msr_misc_feature_control = 1, .has_msr_misc_pwr_mgmt = 1, @@ -1028,12 +1045,14 @@ static const struct platform_data turbostat_pdata[] = { { INTEL_EMERALDRAPIDS_X, &spr_features }, { INTEL_GRANITERAPIDS_X, &spr_features }, { INTEL_GRANITERAPIDS_D, &spr_features }, + { INTEL_PANTHERCOVE_X, &dmr_features }, { INTEL_LAKEFIELD, &cnl_features }, { INTEL_ALDERLAKE, &adl_features }, { INTEL_ALDERLAKE_L, &adl_features }, { INTEL_RAPTORLAKE, &adl_features }, { INTEL_RAPTORLAKE_P, &adl_features }, { INTEL_RAPTORLAKE_S, &adl_features }, + { INTEL_BARTLETTLAKE, &adl_features }, { INTEL_METEORLAKE, &adl_features }, { INTEL_METEORLAKE_L, &adl_features }, { INTEL_ARROWLAKE_H, &adl_features }, @@ -1072,7 +1091,6 @@ void probe_platform_features(unsigned int family, unsigned int model) { int i; - if (authentic_amd || hygon_genuine) { /* fallback to default features on unsupported models */ force_load++; @@ -1106,8 +1124,7 @@ end: if (platform) return; - fprintf(stderr, "Unsupported platform detected.\n" - "\tSee RUN THE LATEST VERSION on turbostat(8)\n"); + fprintf(stderr, "Unsupported platform detected.\n\tSee RUN THE LATEST VERSION on turbostat(8)\n"); exit(1); } @@ -1127,7 +1144,8 @@ char *progname; #define CPU_SUBSET_MAXCPUS 8192 /* need to use before probe... */ cpu_set_t *cpu_present_set, *cpu_possible_set, *cpu_effective_set, *cpu_allowed_set, *cpu_affinity_set, *cpu_subset; -size_t cpu_present_setsize, cpu_possible_setsize, cpu_effective_setsize, cpu_allowed_setsize, cpu_affinity_setsize, cpu_subset_size; +size_t cpu_present_setsize, cpu_possible_setsize, cpu_effective_setsize, cpu_allowed_setsize, cpu_affinity_setsize, + cpu_subset_size; #define MAX_ADDED_THREAD_COUNTERS 24 #define MAX_ADDED_CORE_COUNTERS 8 #define MAX_ADDED_PACKAGE_COUNTERS 16 @@ -2140,13 +2158,20 @@ int get_msr_fd(int cpu) if (fd) return fd; - +#if defined(ANDROID) + sprintf(pathname, "/dev/msr%d", cpu); +#else sprintf(pathname, "/dev/cpu/%d/msr", cpu); +#endif fd = open(pathname, O_RDONLY); if (fd < 0) +#if defined(ANDROID) + err(-1, "%s open failed, try chown or chmod +r /dev/msr*, " + "or run with --no-msr, or run as root", pathname); +#else err(-1, "%s open failed, try chown or chmod +r /dev/cpu/*/msr, " "or run with --no-msr, or run as root", pathname); - +#endif fd_percpu[cpu] = fd; return fd; @@ -2215,32 +2240,52 @@ int get_msr(int cpu, off_t offset, unsigned long long *msr) return 0; } -int probe_rapl_msr(int cpu, off_t offset, int index) +int add_msr_counter(int cpu, off_t offset) { ssize_t retval; unsigned long long value; - assert(!no_msr); + if (no_msr) + return -1; + + if (!offset) + return -1; retval = pread(get_msr_fd(cpu), &value, sizeof(value), offset); /* if the read failed, the probe fails */ if (retval != sizeof(value)) - return 1; + return -1; + + if (value == 0) + return 0; + + return 1; +} + +int add_rapl_msr_counter(int cpu, const struct rapl_counter_arch_info *cai) +{ + int ret; + + if (!(platform->rapl_msrs & cai->feature_mask)) + return -1; + + ret = add_msr_counter(cpu, cai->msr); + if (ret < 0) + return -1; - /* If an Energy Status Counter MSR returns 0, the probe fails */ - switch (index) { + switch (cai->rci_index) { case RAPL_RCI_INDEX_ENERGY_PKG: case RAPL_RCI_INDEX_ENERGY_CORES: case RAPL_RCI_INDEX_DRAM: case RAPL_RCI_INDEX_GFX: case RAPL_RCI_INDEX_ENERGY_PLATFORM: - if (value == 0) + if (ret == 0) return 1; } /* PKG,DRAM_PERF_STATUS MSRs, can return any value */ - return 0; + return 1; } /* Convert CPU ID to domain ID for given added perf counter. */ @@ -2327,8 +2372,7 @@ void help(void) " degrees Celsius\n" " -h, --help\n" " print this help message\n" - " -v, --version\n" - " print version information\n\nFor more help, run \"man turbostat\"\n"); + " -v, --version\n\t\tprint version information\n\nFor more help, run \"man turbostat\"\n"); } /* @@ -2644,7 +2688,7 @@ void print_header(char *delim) if (DO_BIC(BIC_SYS_LPI)) outp += sprintf(outp, "%sSYS%%LPI", (printed++ ? delim : "")); - if (platform->rapl_msrs && !rapl_joules) { + if (!rapl_joules) { if (DO_BIC(BIC_PkgWatt)) outp += sprintf(outp, "%sPkgWatt", (printed++ ? delim : "")); if (DO_BIC(BIC_CorWatt) && !platform->has_per_core_rapl) @@ -2657,7 +2701,7 @@ void print_header(char *delim) outp += sprintf(outp, "%sPKG_%%", (printed++ ? delim : "")); if (DO_BIC(BIC_RAM__)) outp += sprintf(outp, "%sRAM_%%", (printed++ ? delim : "")); - } else if (platform->rapl_msrs && rapl_joules) { + } else { if (DO_BIC(BIC_Pkg_J)) outp += sprintf(outp, "%sPkg_J", (printed++ ? delim : "")); if (DO_BIC(BIC_Cor_J) && !platform->has_per_core_rapl) @@ -3943,7 +3987,6 @@ void compute_average(struct thread_data *t, struct core_data *c, struct pkg_data if (average.threads.nmi_count > 9999999) sums_need_wide_columns = 1; - average.cores.c3 /= topo.allowed_cores; average.cores.c6 /= topo.allowed_cores; average.cores.c7 /= topo.allowed_cores; @@ -4766,6 +4809,37 @@ unsigned long pmt_read_counter(struct pmt_counter *ppmt, unsigned int domain_id) return (value & value_mask) >> value_shift; } +/* Rapl domain enumeration helpers */ +static inline int get_rapl_num_domains(void) +{ + int num_packages = topo.max_package_id + 1; + int num_cores_per_package; + int num_cores; + + if (!platform->has_per_core_rapl) + return num_packages; + + num_cores_per_package = topo.max_core_id + 1; + num_cores = num_cores_per_package * num_packages; + + return num_cores; +} + +static inline int get_rapl_domain_id(int cpu) +{ + int nr_cores_per_package = topo.max_core_id + 1; + int rapl_core_id; + + if (!platform->has_per_core_rapl) + return cpus[cpu].physical_package_id; + + /* Compute the system-wide unique core-id for @cpu */ + rapl_core_id = cpus[cpu].physical_core_id; + rapl_core_id += cpus[cpu].physical_package_id * nr_cores_per_package; + + return rapl_core_id; +} + /* * get_counters(...) * migrate to cpu @@ -4821,7 +4895,7 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p) goto done; if (platform->has_per_core_rapl) { - status = get_rapl_counters(cpu, c->core_id, c, p); + status = get_rapl_counters(cpu, get_rapl_domain_id(cpu), c, p); if (status != 0) return status; } @@ -4887,7 +4961,7 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p) p->sys_lpi = cpuidle_cur_sys_lpi_us; if (!platform->has_per_core_rapl) { - status = get_rapl_counters(cpu, p->package_id, c, p); + status = get_rapl_counters(cpu, get_rapl_domain_id(cpu), c, p); if (status != 0) return status; } @@ -6476,8 +6550,11 @@ void check_dev_msr() if (no_msr) return; - +#if defined(ANDROID) + sprintf(pathname, "/dev/msr%d", base_cpu); +#else sprintf(pathname, "/dev/cpu/%d/msr", base_cpu); +#endif if (stat(pathname, &sb)) if (system("/sbin/modprobe msr > /dev/null 2>&1")) no_msr = 1; @@ -6527,7 +6604,11 @@ void check_msr_permission(void) failed += check_for_cap_sys_rawio(); /* test file permissions */ +#if defined(ANDROID) + sprintf(pathname, "/dev/msr%d", base_cpu); +#else sprintf(pathname, "/dev/cpu/%d/msr", base_cpu); +#endif if (euidaccess(pathname, R_OK)) { failed++; } @@ -6737,8 +6818,10 @@ static void probe_intel_uncore_frequency_cluster(void) * This allows "--show/--hide UncMHz" to be effective for * the clustered MHz counters, as a group. */ - if BIC_IS_ENABLED(BIC_UNCORE_MHZ) - add_counter(0, path, name_buf, 0, SCOPE_PACKAGE, COUNTER_K2M, FORMAT_AVERAGE, 0, package_id); + if BIC_IS_ENABLED + (BIC_UNCORE_MHZ) + add_counter(0, path, name_buf, 0, SCOPE_PACKAGE, COUNTER_K2M, FORMAT_AVERAGE, 0, + package_id); if (quiet) continue; @@ -6810,17 +6893,21 @@ static void probe_graphics(void) else goto next; - set_graphics_fp("/sys/class/drm/card0/device/tile0/gt0/gtidle/idle_residency_ms", gt0_is_gt ? GFX_rc6 : SAM_mc6); + set_graphics_fp("/sys/class/drm/card0/device/tile0/gt0/gtidle/idle_residency_ms", + gt0_is_gt ? GFX_rc6 : SAM_mc6); set_graphics_fp("/sys/class/drm/card0/device/tile0/gt0/freq0/cur_freq", gt0_is_gt ? GFX_MHz : SAM_MHz); - set_graphics_fp("/sys/class/drm/card0/device/tile0/gt0/freq0/act_freq", gt0_is_gt ? GFX_ACTMHz : SAM_ACTMHz); + set_graphics_fp("/sys/class/drm/card0/device/tile0/gt0/freq0/act_freq", + gt0_is_gt ? GFX_ACTMHz : SAM_ACTMHz); - set_graphics_fp("/sys/class/drm/card0/device/tile0/gt1/gtidle/idle_residency_ms", gt0_is_gt ? SAM_mc6 : GFX_rc6); + set_graphics_fp("/sys/class/drm/card0/device/tile0/gt1/gtidle/idle_residency_ms", + gt0_is_gt ? SAM_mc6 : GFX_rc6); set_graphics_fp("/sys/class/drm/card0/device/tile0/gt1/freq0/cur_freq", gt0_is_gt ? SAM_MHz : GFX_MHz); - set_graphics_fp("/sys/class/drm/card0/device/tile0/gt1/freq0/act_freq", gt0_is_gt ? SAM_ACTMHz : GFX_ACTMHz); + set_graphics_fp("/sys/class/drm/card0/device/tile0/gt1/freq0/act_freq", + gt0_is_gt ? SAM_ACTMHz : GFX_ACTMHz); goto end; } @@ -7256,6 +7343,9 @@ void rapl_probe_intel(void) else bic_enabled &= ~bic_joules_bits; + if (!platform->rapl_msrs || no_msr) + return; + if (!(platform->rapl_msrs & RAPL_PKG_PERF_STATUS)) bic_enabled &= ~BIC_PKG__; if (!(platform->rapl_msrs & RAPL_DRAM_PERF_STATUS)) @@ -7306,6 +7396,9 @@ void rapl_probe_amd(void) else bic_enabled &= ~bic_joules_bits; + if (!platform->rapl_msrs || no_msr) + return; + if (get_msr(base_cpu, MSR_RAPL_PWR_UNIT, &msr)) return; @@ -7332,6 +7425,158 @@ void print_power_limit_msr(int cpu, unsigned long long msr, char *label) return; } +static int fread_int(char *path, int *val) +{ + FILE *filep; + int ret; + + filep = fopen(path, "r"); + if (!filep) + return -1; + + ret = fscanf(filep, "%d", val); + fclose(filep); + return ret; +} + +static int fread_ull(char *path, unsigned long long *val) +{ + FILE *filep; + int ret; + + filep = fopen(path, "r"); + if (!filep) + return -1; + + ret = fscanf(filep, "%llu", val); + fclose(filep); + return ret; +} + +static int fread_str(char *path, char *buf, int size) +{ + FILE *filep; + int ret; + char *cp; + + filep = fopen(path, "r"); + if (!filep) + return -1; + + ret = fread(buf, 1, size, filep); + fclose(filep); + + /* replace '\n' with '\0' */ + cp = strchr(buf, '\n'); + if (cp != NULL) + *cp = '\0'; + + return ret; +} + +#define PATH_RAPL_SYSFS "/sys/class/powercap" + +static int dump_one_domain(char *domain_path) +{ + char path[PATH_MAX]; + char str[PATH_MAX]; + unsigned long long val; + int constraint; + int enable; + int ret; + + snprintf(path, PATH_MAX, "%s/name", domain_path); + ret = fread_str(path, str, PATH_MAX); + if (ret <= 0) + return -1; + + fprintf(outf, "%s: %s", domain_path + strlen(PATH_RAPL_SYSFS) + 1, str); + + snprintf(path, PATH_MAX, "%s/enabled", domain_path); + ret = fread_int(path, &enable); + if (ret <= 0) + return -1; + + if (!enable) { + fputs(" disabled\n", outf); + return 0; + } + + for (constraint = 0;; constraint++) { + snprintf(path, PATH_MAX, "%s/constraint_%d_time_window_us", domain_path, constraint); + ret = fread_ull(path, &val); + if (ret <= 0) + break; + + if (val > 1000000) + fprintf(outf, " %0.1fs", (double)val / 1000000); + else if (val > 1000) + fprintf(outf, " %0.1fms", (double)val / 1000); + else + fprintf(outf, " %0.1fus", (double)val); + + snprintf(path, PATH_MAX, "%s/constraint_%d_power_limit_uw", domain_path, constraint); + ret = fread_ull(path, &val); + if (ret > 0 && val) + fprintf(outf, ":%lluW", val / 1000000); + + snprintf(path, PATH_MAX, "%s/constraint_%d_max_power_uw", domain_path, constraint); + ret = fread_ull(path, &val); + if (ret > 0 && val) + fprintf(outf, ",max:%lluW", val / 1000000); + } + fputc('\n', outf); + + return 0; +} + +static int print_rapl_sysfs(void) +{ + DIR *dir, *cdir; + struct dirent *entry, *centry; + char path[PATH_MAX]; + char str[PATH_MAX]; + + if ((dir = opendir(PATH_RAPL_SYSFS)) == NULL) { + warn("open %s failed", PATH_RAPL_SYSFS); + return 1; + } + + while ((entry = readdir(dir)) != NULL) { + if (strlen(entry->d_name) > 100) + continue; + + if (strncmp(entry->d_name, "intel-rapl", strlen("intel-rapl"))) + continue; + + snprintf(path, PATH_MAX, "%s/%s/name", PATH_RAPL_SYSFS, entry->d_name); + + /* Parse top level domains first, including package and psys */ + fread_str(path, str, PATH_MAX); + if (strncmp(str, "package", strlen("package")) && strncmp(str, "psys", strlen("psys"))) + continue; + + snprintf(path, PATH_MAX, "%s/%s", PATH_RAPL_SYSFS, entry->d_name); + if ((cdir = opendir(path)) == NULL) { + perror("opendir() error"); + return 1; + } + + dump_one_domain(path); + + while ((centry = readdir(cdir)) != NULL) { + if (strncmp(centry->d_name, "intel-rapl", strlen("intel-rapl"))) + continue; + snprintf(path, PATH_MAX, "%s/%s/%s", PATH_RAPL_SYSFS, entry->d_name, centry->d_name); + dump_one_domain(path); + } + closedir(cdir); + } + + closedir(dir); + return 0; +} + int print_rapl(struct thread_data *t, struct core_data *c, struct pkg_data *p) { unsigned long long msr; @@ -7458,9 +7703,6 @@ int print_rapl(struct thread_data *t, struct core_data *c, struct pkg_data *p) */ void probe_rapl(void) { - if (!platform->rapl_msrs || no_msr) - return; - if (genuine_intel) rapl_probe_intel(); if (authentic_amd || hygon_genuine) @@ -7469,6 +7711,11 @@ void probe_rapl(void) if (quiet) return; + print_rapl_sysfs(); + + if (!platform->rapl_msrs || no_msr) + return; + for_all_cpus(print_rapl, ODD_COUNTERS); } @@ -7801,44 +8048,42 @@ static int has_instr_count_access(void) return has_access; } -int add_rapl_perf_counter_(int cpu, struct rapl_counter_info_t *rci, const struct rapl_counter_arch_info *cai, - double *scale_, enum rapl_unit *unit_) +int add_rapl_perf_counter(int cpu, struct rapl_counter_info_t *rci, const struct rapl_counter_arch_info *cai, + double *scale_, enum rapl_unit *unit_) { + int ret = -1; + if (no_perf) return -1; + if (!cai->perf_name) + return -1; + const double scale = read_perf_scale(cai->perf_subsys, cai->perf_name); if (scale == 0.0) - return -1; + goto end; const enum rapl_unit unit = read_perf_rapl_unit(cai->perf_subsys, cai->perf_name); if (unit == RAPL_UNIT_INVALID) - return -1; + goto end; const unsigned int rapl_type = read_perf_type(cai->perf_subsys); const unsigned int rapl_energy_pkg_config = read_perf_config(cai->perf_subsys, cai->perf_name); - const int fd_counter = - open_perf_counter(cpu, rapl_type, rapl_energy_pkg_config, rci->fd_perf, PERF_FORMAT_GROUP); - if (fd_counter == -1) - return -1; + ret = open_perf_counter(cpu, rapl_type, rapl_energy_pkg_config, rci->fd_perf, PERF_FORMAT_GROUP); + if (ret == -1) + goto end; /* If it's the first counter opened, make it a group descriptor */ if (rci->fd_perf == -1) - rci->fd_perf = fd_counter; + rci->fd_perf = ret; *scale_ = scale; *unit_ = unit; - return fd_counter; -} - -int add_rapl_perf_counter(int cpu, struct rapl_counter_info_t *rci, const struct rapl_counter_arch_info *cai, - double *scale, enum rapl_unit *unit) -{ - int ret = add_rapl_perf_counter_(cpu, rci, cai, scale, unit); +end: if (debug >= 2) fprintf(stderr, "%s: %d (cpu: %d)\n", __func__, ret, cpu); @@ -7863,7 +8108,7 @@ void linux_perf_init(void) void rapl_perf_init(void) { - const unsigned int num_domains = (platform->has_per_core_rapl ? topo.max_core_id : topo.max_package_id) + 1; + const unsigned int num_domains = get_rapl_num_domains(); bool *domain_visited = calloc(num_domains, sizeof(bool)); rapl_counter_info_perdomain = calloc(num_domains, sizeof(*rapl_counter_info_perdomain)); @@ -7896,6 +8141,9 @@ void rapl_perf_init(void) enum rapl_unit unit; unsigned int next_domain; + if (!BIC_IS_ENABLED(cai->bic)) + continue; + memset(domain_visited, 0, num_domains * sizeof(*domain_visited)); for (int cpu = 0; cpu < topo.max_cpu_num + 1; ++cpu) { @@ -7904,8 +8152,7 @@ void rapl_perf_init(void) continue; /* Skip already seen and handled RAPL domains */ - next_domain = - platform->has_per_core_rapl ? cpus[cpu].physical_core_id : cpus[cpu].physical_package_id; + next_domain = get_rapl_domain_id(cpu); assert(next_domain < num_domains); @@ -7919,27 +8166,37 @@ void rapl_perf_init(void) struct rapl_counter_info_t *rci = &rapl_counter_info_perdomain[next_domain]; - /* Check if the counter is enabled and accessible */ - if (BIC_IS_ENABLED(cai->bic) && (platform->rapl_msrs & cai->feature_mask)) { + /* + * rapl_counter_arch_infos[] can have multiple entries describing the same + * counter, due to the difference from different platforms/Vendors. + * E.g. rapl_counter_arch_infos[0] and rapl_counter_arch_infos[1] share the + * same perf_subsys and perf_name, but with different MSR address. + * rapl_counter_arch_infos[0] is for Intel and rapl_counter_arch_infos[1] + * is for AMD. + * In this case, it is possible that multiple rapl_counter_arch_infos[] + * entries are probed just because their perf/msr is duplicate and valid. + * + * Thus need a check to avoid re-probe the same counters. + */ + if (rci->source[cai->rci_index] != COUNTER_SOURCE_NONE) + break; - /* Use perf API for this counter */ - if (!no_perf && cai->perf_name - && add_rapl_perf_counter(cpu, rci, cai, &scale, &unit) != -1) { - rci->source[cai->rci_index] = COUNTER_SOURCE_PERF; - rci->scale[cai->rci_index] = scale * cai->compat_scale; - rci->unit[cai->rci_index] = unit; - rci->flags[cai->rci_index] = cai->flags; - - /* Use MSR for this counter */ - } else if (!no_msr && cai->msr && probe_rapl_msr(cpu, cai->msr, cai->rci_index) == 0) { - rci->source[cai->rci_index] = COUNTER_SOURCE_MSR; - rci->msr[cai->rci_index] = cai->msr; - rci->msr_mask[cai->rci_index] = cai->msr_mask; - rci->msr_shift[cai->rci_index] = cai->msr_shift; - rci->unit[cai->rci_index] = RAPL_UNIT_JOULES; - rci->scale[cai->rci_index] = *cai->platform_rapl_msr_scale * cai->compat_scale; - rci->flags[cai->rci_index] = cai->flags; - } + /* Use perf API for this counter */ + if (add_rapl_perf_counter(cpu, rci, cai, &scale, &unit) != -1) { + rci->source[cai->rci_index] = COUNTER_SOURCE_PERF; + rci->scale[cai->rci_index] = scale * cai->compat_scale; + rci->unit[cai->rci_index] = unit; + rci->flags[cai->rci_index] = cai->flags; + + /* Use MSR for this counter */ + } else if (add_rapl_msr_counter(cpu, cai) >= 0) { + rci->source[cai->rci_index] = COUNTER_SOURCE_MSR; + rci->msr[cai->rci_index] = cai->msr; + rci->msr_mask[cai->rci_index] = cai->msr_mask; + rci->msr_shift[cai->rci_index] = cai->msr_shift; + rci->unit[cai->rci_index] = RAPL_UNIT_JOULES; + rci->scale[cai->rci_index] = *cai->platform_rapl_msr_scale * cai->compat_scale; + rci->flags[cai->rci_index] = cai->flags; } if (rci->source[cai->rci_index] != COUNTER_SOURCE_NONE) @@ -7972,65 +8229,63 @@ int *get_cstate_perf_group_fd(struct cstate_counter_info_t *cci, const char *gro return NULL; } -int add_cstate_perf_counter_(int cpu, struct cstate_counter_info_t *cci, const struct cstate_counter_arch_info *cai) +int add_cstate_perf_counter(int cpu, struct cstate_counter_info_t *cci, const struct cstate_counter_arch_info *cai) { + int ret = -1; + if (no_perf) return -1; + if (!cai->perf_name) + return -1; + int *pfd_group = get_cstate_perf_group_fd(cci, cai->perf_subsys); if (pfd_group == NULL) - return -1; + goto end; const unsigned int type = read_perf_type(cai->perf_subsys); const unsigned int config = read_perf_config(cai->perf_subsys, cai->perf_name); - const int fd_counter = open_perf_counter(cpu, type, config, *pfd_group, PERF_FORMAT_GROUP); + ret = open_perf_counter(cpu, type, config, *pfd_group, PERF_FORMAT_GROUP); - if (fd_counter == -1) - return -1; + if (ret == -1) + goto end; /* If it's the first counter opened, make it a group descriptor */ if (*pfd_group == -1) - *pfd_group = fd_counter; - - return fd_counter; -} - -int add_cstate_perf_counter(int cpu, struct cstate_counter_info_t *cci, const struct cstate_counter_arch_info *cai) -{ - int ret = add_cstate_perf_counter_(cpu, cci, cai); + *pfd_group = ret; +end: if (debug >= 2) fprintf(stderr, "%s: %d (cpu: %d)\n", __func__, ret, cpu); return ret; } -int add_msr_perf_counter_(int cpu, struct msr_counter_info_t *cci, const struct msr_counter_arch_info *cai) +int add_msr_perf_counter(int cpu, struct msr_counter_info_t *cci, const struct msr_counter_arch_info *cai) { + int ret = -1; + if (no_perf) return -1; + if (!cai->perf_name) + return -1; + const unsigned int type = read_perf_type(cai->perf_subsys); const unsigned int config = read_perf_config(cai->perf_subsys, cai->perf_name); - const int fd_counter = open_perf_counter(cpu, type, config, cci->fd_perf, PERF_FORMAT_GROUP); + ret = open_perf_counter(cpu, type, config, cci->fd_perf, PERF_FORMAT_GROUP); - if (fd_counter == -1) - return -1; + if (ret == -1) + goto end; /* If it's the first counter opened, make it a group descriptor */ if (cci->fd_perf == -1) - cci->fd_perf = fd_counter; - - return fd_counter; -} - -int add_msr_perf_counter(int cpu, struct msr_counter_info_t *cci, const struct msr_counter_arch_info *cai) -{ - int ret = add_msr_perf_counter_(cpu, cci, cai); + cci->fd_perf = ret; +end: if (debug) fprintf(stderr, "%s: %s/%s: %d (cpu: %d)\n", __func__, cai->perf_subsys, cai->perf_name, ret, cpu); @@ -8064,12 +8319,12 @@ void msr_perf_init_(void) if (cai->needed) { /* Use perf API for this counter */ - if (!no_perf && cai->perf_name && add_msr_perf_counter(cpu, cci, cai) != -1) { + if (add_msr_perf_counter(cpu, cci, cai) != -1) { cci->source[cai->rci_index] = COUNTER_SOURCE_PERF; cai->present = true; /* User MSR for this counter */ - } else if (!no_msr && cai->msr && probe_rapl_msr(cpu, cai->msr, cai->rci_index) == 0) { + } else if (add_msr_counter(cpu, cai->msr) >= 0) { cci->source[cai->rci_index] = COUNTER_SOURCE_MSR; cci->msr[cai->rci_index] = cai->msr; cci->msr_mask[cai->rci_index] = cai->msr_mask; @@ -8177,13 +8432,13 @@ void cstate_perf_init_(bool soft_c1) if (counter_needed && counter_supported) { /* Use perf API for this counter */ - if (!no_perf && cai->perf_name && add_cstate_perf_counter(cpu, cci, cai) != -1) { + if (add_cstate_perf_counter(cpu, cci, cai) != -1) { cci->source[cai->rci_index] = COUNTER_SOURCE_PERF; /* User MSR for this counter */ - } else if (!no_msr && cai->msr && pkg_cstate_limit >= cai->pkg_cstate_limit - && probe_rapl_msr(cpu, cai->msr, cai->rci_index) == 0) { + } else if (pkg_cstate_limit >= cai->pkg_cstate_limit + && add_msr_counter(cpu, cai->msr) >= 0) { cci->source[cai->rci_index] = COUNTER_SOURCE_MSR; cci->msr[cai->rci_index] = cai->msr; } @@ -9044,15 +9299,14 @@ int added_perf_counters_init_(struct perf_counter_info *pinfo) perf_device = "cpu_atom"; break; - default: /* Don't change, we will probably fail and report a problem soon. */ + default: /* Don't change, we will probably fail and report a problem soon. */ break; } } perf_type = read_perf_type(perf_device); if (perf_type == (unsigned int)-1) { - warnx("%s: perf/%s/%s: failed to read %s", - __func__, perf_device, pinfo->event, "type"); + warnx("%s: perf/%s/%s: failed to read %s", __func__, perf_device, pinfo->event, "type"); continue; } @@ -9154,7 +9408,7 @@ struct pmt_mmio *pmt_mmio_open(unsigned int target_guid) return NULL; } - for ( ; entry != NULL; entry = pmt_diriter_next(&pmt_iter)) { + for (; entry != NULL; entry = pmt_diriter_next(&pmt_iter)) { if (fstatat(dirfd(pmt_iter.dir), entry->d_name, &st, 0) == -1) break; @@ -9594,7 +9848,7 @@ int get_and_dump_counters(void) void print_version() { - fprintf(outf, "turbostat version 2025.04.06 - Len Brown <lenb@kernel.org>\n"); + fprintf(outf, "turbostat version 2025.06.08 - Len Brown <lenb@kernel.org>\n"); } #define COMMAND_LINE_SIZE 2048 @@ -10050,7 +10304,7 @@ void parse_add_command_pmt(char *add_command) unsigned int lsb; unsigned int msb; unsigned int guid; - unsigned int seq = 0; /* By default, pick first file in a sequence with a given GUID. */ + unsigned int seq = 0; /* By default, pick first file in a sequence with a given GUID. */ unsigned int domain_id; enum counter_scope scope = 0; enum pmt_datatype type = PMT_TYPE_RAW; diff --git a/tools/testing/selftests/drivers/net/hw/rss_ctx.py b/tools/testing/selftests/drivers/net/hw/rss_ctx.py index ca60ae325c22..7bb552f8b182 100755 --- a/tools/testing/selftests/drivers/net/hw/rss_ctx.py +++ b/tools/testing/selftests/drivers/net/hw/rss_ctx.py @@ -747,6 +747,62 @@ def test_rss_ntuple_addition(cfg): 'noise' : (0,) }) +def test_rss_default_context_rule(cfg): + """ + Allocate a port, direct this port to context 0, then create a new RSS + context and steer all TCP traffic to it (context 1). Verify that: + * Traffic to the specific port continues to use queues of the main + context (0/1). + * Traffic to any other TCP port is redirected to the new context + (queues 2/3). + """ + + require_ntuple(cfg) + + queue_cnt = len(_get_rx_cnts(cfg)) + if queue_cnt < 4: + try: + ksft_pr(f"Increasing queue count {queue_cnt} -> 4") + ethtool(f"-L {cfg.ifname} combined 4") + defer(ethtool, f"-L {cfg.ifname} combined {queue_cnt}") + except Exception as exc: + raise KsftSkipEx("Not enough queues for the test") from exc + + # Use queues 0 and 1 for the main context + ethtool(f"-X {cfg.ifname} equal 2") + defer(ethtool, f"-X {cfg.ifname} default") + + # Create a new RSS context that uses queues 2 and 3 + ctx_id = ethtool_create(cfg, "-X", "context new start 2 equal 2") + defer(ethtool, f"-X {cfg.ifname} context {ctx_id} delete") + + # Generic low-priority rule: redirect all TCP traffic to the new context. + # Give it an explicit higher location number (lower priority). + flow_generic = f"flow-type tcp{cfg.addr_ipver} dst-ip {cfg.addr} context {ctx_id} loc 1" + ethtool(f"-N {cfg.ifname} {flow_generic}") + defer(ethtool, f"-N {cfg.ifname} delete 1") + + # Specific high-priority rule for a random port that should stay on context 0. + # Assign loc 0 so it is evaluated before the generic rule. + port_main = rand_port() + flow_main = f"flow-type tcp{cfg.addr_ipver} dst-ip {cfg.addr} dst-port {port_main} context 0 loc 0" + ethtool(f"-N {cfg.ifname} {flow_main}") + defer(ethtool, f"-N {cfg.ifname} delete 0") + + _ntuple_rule_check(cfg, 1, ctx_id) + + # Verify that traffic matching the specific rule still goes to queues 0/1 + _send_traffic_check(cfg, port_main, "context 0", + { 'target': (0, 1), + 'empty' : (2, 3) }) + + # And that traffic for any other port is steered to the new context + port_other = rand_port() + _send_traffic_check(cfg, port_other, f"context {ctx_id}", + { 'target': (2, 3), + 'noise' : (0, 1) }) + + def main() -> None: with NetDrvEpEnv(__file__, nsim_test=False) as cfg: cfg.context_cnt = None @@ -760,7 +816,8 @@ def main() -> None: test_rss_context_overlap, test_rss_context_overlap2, test_rss_context_out_of_order, test_rss_context4_create_with_cfg, test_flow_add_context_missing, - test_delete_rss_context_busy, test_rss_ntuple_addition], + test_delete_rss_context_busy, test_rss_ntuple_addition, + test_rss_default_context_rule], args=(cfg, )) ksft_exit() diff --git a/tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c b/tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c index a36a7e2db434..b4d22b3ab7cc 100644 --- a/tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c +++ b/tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c @@ -22,7 +22,8 @@ #include "gic.h" #include "vgic.h" -static const uint64_t CVAL_MAX = ~0ULL; +/* Depends on counter width. */ +static uint64_t CVAL_MAX; /* tval is a signed 32-bit int. */ static const int32_t TVAL_MAX = INT32_MAX; static const int32_t TVAL_MIN = INT32_MIN; @@ -30,8 +31,8 @@ static const int32_t TVAL_MIN = INT32_MIN; /* After how much time we say there is no IRQ. */ static const uint32_t TIMEOUT_NO_IRQ_US = 50000; -/* A nice counter value to use as the starting one for most tests. */ -static const uint64_t DEF_CNT = (CVAL_MAX / 2); +/* Counter value to use as the starting one for most tests. Set to CVAL_MAX/2 */ +static uint64_t DEF_CNT; /* Number of runs. */ static const uint32_t NR_TEST_ITERS_DEF = 5; @@ -191,8 +192,8 @@ static void set_tval_irq(enum arch_timer timer, uint64_t tval_cycles, { atomic_set(&shared_data.handled, 0); atomic_set(&shared_data.spurious, 0); - timer_set_ctl(timer, ctl); timer_set_tval(timer, tval_cycles); + timer_set_ctl(timer, ctl); } static void set_xval_irq(enum arch_timer timer, uint64_t xval, uint32_t ctl, @@ -732,12 +733,6 @@ static void test_move_counters_ahead_of_timers(enum arch_timer timer) test_set_cnt_after_tval(timer, 0, tval, (uint64_t) tval + 1, wm); } - - for (i = 0; i < ARRAY_SIZE(sleep_method); i++) { - sleep_method_t sm = sleep_method[i]; - - test_set_cnt_after_cval_no_irq(timer, 0, DEF_CNT, CVAL_MAX, sm); - } } /* @@ -849,17 +844,17 @@ static void guest_code(enum arch_timer timer) GUEST_DONE(); } +static cpu_set_t default_cpuset; + static uint32_t next_pcpu(void) { uint32_t max = get_nprocs(); uint32_t cur = sched_getcpu(); uint32_t next = cur; - cpu_set_t cpuset; + cpu_set_t cpuset = default_cpuset; TEST_ASSERT(max > 1, "Need at least two physical cpus"); - sched_getaffinity(0, sizeof(cpuset), &cpuset); - do { next = (next + 1) % CPU_SETSIZE; } while (!CPU_ISSET(next, &cpuset)); @@ -975,6 +970,8 @@ static void test_vm_create(struct kvm_vm **vm, struct kvm_vcpu **vcpu, test_init_timer_irq(*vm, *vcpu); vgic_v3_setup(*vm, 1, 64); sync_global_to_guest(*vm, test_args); + sync_global_to_guest(*vm, CVAL_MAX); + sync_global_to_guest(*vm, DEF_CNT); } static void test_print_help(char *name) @@ -986,7 +983,7 @@ static void test_print_help(char *name) pr_info("\t-b: Test both physical and virtual timers (default: true)\n"); pr_info("\t-l: Delta (in ms) used for long wait time test (default: %u)\n", LONG_WAIT_TEST_MS); - pr_info("\t-l: Delta (in ms) used for wait times (default: %u)\n", + pr_info("\t-w: Delta (in ms) used for wait times (default: %u)\n", WAIT_TEST_MS); pr_info("\t-p: Test physical timer (default: true)\n"); pr_info("\t-v: Test virtual timer (default: true)\n"); @@ -1035,6 +1032,17 @@ static bool parse_args(int argc, char *argv[]) return false; } +static void set_counter_defaults(void) +{ + const uint64_t MIN_ROLLOVER_SECS = 40ULL * 365 * 24 * 3600; + uint64_t freq = read_sysreg(CNTFRQ_EL0); + uint64_t width = ilog2(MIN_ROLLOVER_SECS * freq); + + width = clamp(width, 56, 64); + CVAL_MAX = GENMASK_ULL(width - 1, 0); + DEF_CNT = CVAL_MAX / 2; +} + int main(int argc, char *argv[]) { struct kvm_vcpu *vcpu; @@ -1046,6 +1054,9 @@ int main(int argc, char *argv[]) if (!parse_args(argc, argv)) exit(KSFT_SKIP); + sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset); + set_counter_defaults(); + if (test_args.test_virtual) { test_vm_create(&vm, &vcpu, VIRTUAL); test_run(vm, vcpu); diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c index dcdd5bb20f3d..e80deac1436b 100644 --- a/tools/testing/selftests/mm/ksm_tests.c +++ b/tools/testing/selftests/mm/ksm_tests.c @@ -58,40 +58,12 @@ int debug; static int ksm_write_sysfs(const char *file_path, unsigned long val) { - FILE *f = fopen(file_path, "w"); - - if (!f) { - fprintf(stderr, "f %s\n", file_path); - perror("fopen"); - return 1; - } - if (fprintf(f, "%lu", val) < 0) { - perror("fprintf"); - fclose(f); - return 1; - } - fclose(f); - - return 0; + return write_sysfs(file_path, val); } static int ksm_read_sysfs(const char *file_path, unsigned long *val) { - FILE *f = fopen(file_path, "r"); - - if (!f) { - fprintf(stderr, "f %s\n", file_path); - perror("fopen"); - return 1; - } - if (fscanf(f, "%lu", val) != 1) { - perror("fscanf"); - fclose(f); - return 1; - } - fclose(f); - - return 0; + return read_sysfs(file_path, val); } static void ksm_print_sysfs(void) diff --git a/tools/testing/selftests/mm/merge.c b/tools/testing/selftests/mm/merge.c index c76646cdf6e6..bbae66fc5038 100644 --- a/tools/testing/selftests/mm/merge.c +++ b/tools/testing/selftests/mm/merge.c @@ -2,11 +2,14 @@ #define _GNU_SOURCE #include "../kselftest_harness.h" +#include <fcntl.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/mman.h> +#include <sys/syscall.h> #include <sys/wait.h> +#include <linux/perf_event.h> #include "vm_util.h" FIXTURE(merge) @@ -452,4 +455,44 @@ TEST_F(merge, forked_source_vma) ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr2 + 5 * page_size); } +TEST_F(merge, handle_uprobe_upon_merged_vma) +{ + const size_t attr_sz = sizeof(struct perf_event_attr); + unsigned int page_size = self->page_size; + const char *probe_file = "./foo"; + char *carveout = self->carveout; + struct perf_event_attr attr; + unsigned long type; + void *ptr1, *ptr2; + int fd; + + fd = open(probe_file, O_RDWR|O_CREAT, 0600); + ASSERT_GE(fd, 0); + + ASSERT_EQ(ftruncate(fd, page_size), 0); + ASSERT_EQ(read_sysfs("/sys/bus/event_source/devices/uprobe/type", &type), 0); + + memset(&attr, 0, attr_sz); + attr.size = attr_sz; + attr.type = type; + attr.config1 = (__u64)(long)probe_file; + attr.config2 = 0x0; + + ASSERT_GE(syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0), 0); + + ptr1 = mmap(&carveout[page_size], 10 * page_size, PROT_EXEC, + MAP_PRIVATE | MAP_FIXED, fd, 0); + ASSERT_NE(ptr1, MAP_FAILED); + + ptr2 = mremap(ptr1, page_size, 2 * page_size, + MREMAP_MAYMOVE | MREMAP_FIXED, ptr1 + 5 * page_size); + ASSERT_NE(ptr2, MAP_FAILED); + + ASSERT_NE(mremap(ptr2, page_size, page_size, + MREMAP_MAYMOVE | MREMAP_FIXED, ptr1), MAP_FAILED); + + close(fd); + remove(probe_file); +} + TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c index a41bc1234b37..95b6f043a3cb 100644 --- a/tools/testing/selftests/mm/thuge-gen.c +++ b/tools/testing/selftests/mm/thuge-gen.c @@ -77,7 +77,7 @@ void show(unsigned long ps) system(buf); } -unsigned long read_sysfs(int warn, char *fmt, ...) +unsigned long thuge_read_sysfs(int warn, char *fmt, ...) { char *line = NULL; size_t linelen = 0; @@ -106,7 +106,7 @@ unsigned long read_sysfs(int warn, char *fmt, ...) unsigned long read_free(unsigned long ps) { - return read_sysfs(ps != getpagesize(), + return thuge_read_sysfs(ps != getpagesize(), "/sys/kernel/mm/hugepages/hugepages-%lukB/free_hugepages", ps >> 10); } @@ -195,7 +195,7 @@ void find_pagesizes(void) } globfree(&g); - if (read_sysfs(0, "/proc/sys/kernel/shmmax") < NUM_PAGES * largest) + if (thuge_read_sysfs(0, "/proc/sys/kernel/shmmax") < NUM_PAGES * largest) ksft_exit_fail_msg("Please do echo %lu > /proc/sys/kernel/shmmax", largest * NUM_PAGES); diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c index 61d7bf1f8c62..5492e3f784df 100644 --- a/tools/testing/selftests/mm/vm_util.c +++ b/tools/testing/selftests/mm/vm_util.c @@ -486,3 +486,41 @@ int close_procmap(struct procmap_fd *procmap) { return close(procmap->fd); } + +int write_sysfs(const char *file_path, unsigned long val) +{ + FILE *f = fopen(file_path, "w"); + + if (!f) { + fprintf(stderr, "f %s\n", file_path); + perror("fopen"); + return 1; + } + if (fprintf(f, "%lu", val) < 0) { + perror("fprintf"); + fclose(f); + return 1; + } + fclose(f); + + return 0; +} + +int read_sysfs(const char *file_path, unsigned long *val) +{ + FILE *f = fopen(file_path, "r"); + + if (!f) { + fprintf(stderr, "f %s\n", file_path); + perror("fopen"); + return 1; + } + if (fscanf(f, "%lu", val) != 1) { + perror("fscanf"); + fclose(f); + return 1; + } + fclose(f); + + return 0; +} diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h index adb5d294a220..b8136d12a0f8 100644 --- a/tools/testing/selftests/mm/vm_util.h +++ b/tools/testing/selftests/mm/vm_util.h @@ -88,6 +88,8 @@ int open_procmap(pid_t pid, struct procmap_fd *procmap_out); int query_procmap(struct procmap_fd *procmap); bool find_vma_procmap(struct procmap_fd *procmap, void *address); int close_procmap(struct procmap_fd *procmap); +int write_sysfs(const char *file_path, unsigned long val); +int read_sysfs(const char *file_path, unsigned long *val); static inline int open_self_procmap(struct procmap_fd *procmap_out) { diff --git a/tools/testing/selftests/mount_setattr/mount_setattr_test.c b/tools/testing/selftests/mount_setattr/mount_setattr_test.c index 8b378c91debf..b1e4618399be 100644 --- a/tools/testing/selftests/mount_setattr/mount_setattr_test.c +++ b/tools/testing/selftests/mount_setattr/mount_setattr_test.c @@ -2079,24 +2079,9 @@ TEST_F(mount_setattr, detached_tree_propagation) * means that the device information will be different for any * statx() that was taken from /mnt/A before the mount compared * to one after the mount. - * - * Since we already now that the device information between the - * stx1 and stx2 samples are identical we also now that stx2 and - * stx3 device information will necessarily differ. */ ASSERT_NE(stx1.stx_dev_minor, stx3.stx_dev_minor); - - /* - * If mount propagation worked correctly then the tmpfs mount - * that was created after the mount namespace was unshared will - * have propagated onto /mnt/A in the detached mount tree. - * - * Verify that the device information for stx3 and stx4 are - * identical. It is already established that stx3 is different - * from both stx1 and stx2 sampled before the tmpfs mount was - * done so if stx3 and stx4 are identical the proof is done. - */ - ASSERT_EQ(stx3.stx_dev_minor, stx4.stx_dev_minor); + ASSERT_EQ(stx1.stx_dev_minor, stx4.stx_dev_minor); EXPECT_EQ(close(fd_tree), 0); } diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index ea84b88bcb30..ab996bd22a5f 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -27,6 +27,7 @@ TEST_PROGS += amt.sh TEST_PROGS += unicast_extensions.sh TEST_PROGS += udpgro_fwd.sh TEST_PROGS += udpgro_frglist.sh +TEST_PROGS += nat6to4.sh TEST_PROGS += veth.sh TEST_PROGS += ioam6.sh TEST_PROGS += gro.sh diff --git a/tools/testing/selftests/net/nat6to4.sh b/tools/testing/selftests/net/nat6to4.sh new file mode 100755 index 000000000000..0ee859b622a4 --- /dev/null +++ b/tools/testing/selftests/net/nat6to4.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +NS="ns-peer-$(mktemp -u XXXXXX)" + +ip netns add "${NS}" +ip -netns "${NS}" link set lo up +ip -netns "${NS}" route add default via 127.0.0.2 dev lo + +tc -n "${NS}" qdisc add dev lo ingress +tc -n "${NS}" filter add dev lo ingress prio 4 protocol ip \ + bpf object-file nat6to4.bpf.o section schedcls/egress4/snat4 direct-action + +ip netns exec "${NS}" \ + bash -c 'echo 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789abc | socat - UDP4-DATAGRAM:224.1.0.1:6666,ip-multicast-loop=1' diff --git a/tools/testing/selftests/vDSO/vgetrandom-chacha.S b/tools/testing/selftests/vDSO/vgetrandom-chacha.S index d6e09af7c0a9..a4a82e1c28a9 100644 --- a/tools/testing/selftests/vDSO/vgetrandom-chacha.S +++ b/tools/testing/selftests/vDSO/vgetrandom-chacha.S @@ -11,6 +11,8 @@ #include "../../../../arch/loongarch/vdso/vgetrandom-chacha.S" #elif defined(__powerpc__) || defined(__powerpc64__) #include "../../../../arch/powerpc/kernel/vdso/vgetrandom-chacha.S" +#elif defined(__riscv) && __riscv_xlen == 64 +#include "../../../../arch/riscv/kernel/vdso/vgetrandom-chacha.S" #elif defined(__s390x__) #include "../../../../arch/s390/kernel/vdso64/vgetrandom-chacha.S" #elif defined(__x86_64__) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index 441feb21aa5a..4505b1c31be1 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -932,6 +932,8 @@ static inline void vma_adjust_trans_huge(struct vm_area_struct *vma, (void)next; } +static inline void hugetlb_split(struct vm_area_struct *, unsigned long) {} + static inline void vma_iter_free(struct vma_iterator *vmi) { mas_destroy(&vmi->mas); |