summaryrefslogtreecommitdiff
path: root/drivers/iommu/intel-iommu.c
AgeCommit message (Collapse)Author
2014-07-04iommu/vt-d: Use iommu_group_get_for_dev()Alex Williamson
The IOMMU code now provides a common interface for finding or creating an IOMMU group for a device on PCI buses. Make use of it and remove piles of code. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Cc: David Woodhouse <dwmw2@infradead.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Suppress compiler warningsYijing Wang
suppress compiler warnings: drivers/iommu/intel-iommu.c: In function ‘device_to_iommu’: drivers/iommu/intel-iommu.c:673: warning: ‘segment’ may be used uninitialized in this function drivers/iommu/intel-iommu.c: In function ‘get_domain_for_dev.clone.3’: drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_bus’ may be used uninitialized in this function drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_devfn’ may be used uninitialized in this function Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Remove the useless dma_pte_addrYijing Wang
Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Don't use magic number in dma_pte_superpageJoerg Roedel
Use the already defined DMA_PTE_LARGE_PAGE for testing instead of hardcoding the value again. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Fix reference count in iommu_prepare_isaYijing Wang
Decrease the device reference count avoid memory leak. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Use inline function dma_pte_superpage instead of macrosYijing Wang
Use inline function dma_pte_superpage() instead of macro for better readability. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Clear the redundant assignment for domain->nidYijing Wang
Alloc_domain() will initialize domain->nid to -1. So the initialization for domain->nid in md_domain_init() is redundant, clear it. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/vt-d: Use list_for_each_safe() to simplify codeYijing Wang
Use list_for_each_entry_safe() instead of list_entry() to simplify code. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-06-20iommu/vt-d: fix bug in handling multiple RMRRs for the same PCI deviceJiang Liu
Function dmar_iommu_notify_scope_dev() makes a wrong assumption that there's one RMRR for each PCI device at most, which causes DMA failure on some HP platforms. So enhance dmar_iommu_notify_scope_dev() to handle multiple RMRRs for the same PCI device. Fixbug: https://bugzilla.novell.com/show_bug.cgi?id=879482 Cc: <stable@vger.kernel.org> # 3.15 Reported-by: Tom Mingarelli <thomas.mingarelli@hp.com> Tested-by: Linda Knippers <linda.knippers@hp.com> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-06-04intel-iommu: integrate DMA CMAAkinobu Mita
This adds support for the DMA Contiguous Memory Allocator for intel-iommu. This change enables dma_alloc_coherent() to allocate big contiguous memory. It is achieved in the same way as nommu_dma_ops currently does, i.e. trying to allocate memory by dma_alloc_from_contiguous() and alloc_pages() is used as a fallback. Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Don Dutile <ddutile@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-04-14iommu/vt-d: Fix get_domain_for_dev() handling of upstream PCIe bridgesDavid Woodhouse
Commit 146922ec79 ("iommu/vt-d: Make get_domain_for_dev() take struct device") introduced new variables bridge_bus and bridge_devfn to identify the upstream PCIe to PCI bridge responsible for the given target device. Leaving the original bus/devfn variables to identify the target device itself, now that it is no longer assumed to be PCI and we can no longer trivially find that information. However, the patch failed to correctly use the new variables in all cases; instead using the as-yet-uninitialised 'bus' and 'devfn' variables. Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-04-13iommu/vt-d: fix memory leakage caused by commit ea8ea46Jiang Liu
Commit ea8ea46 "iommu/vt-d: Clean up and fix page table clear/free behaviour" introduces possible leakage of DMA page tables due to: for (pte = page_address(pg); !first_pte_in_page(pte); pte++) { if (dma_pte_present(pte) && !dma_pte_superpage(pte)) freelist = dma_pte_list_pagetables(domain, level - 1, pte, freelist); } For the first pte in a page, first_pte_in_page(pte) will always be true, thus dma_pte_list_pagetables() will never be called and leak DMA page tables if level is bigger than 1. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-28iommu/vt-d: returning free pointer in get_domain_for_dev()Dan Carpenter
If we hit this error condition then we want to return a NULL pointer and not a freed variable. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Include ACPI devices in iommu=ptDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Finally enable translation for non-PCI devicesDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove to_pci_dev() in intel_map_page()David Woodhouse
It might not be... Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove pdev from intel_iommu_attach_device()David Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove pdev from iommu_no_mapping()David Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make domain_add_dev_info() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make domain_remove_one_dev_info() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Rename 'hwdev' variables to 'dev' now that that's the normDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove some pointless to_pci_dev() callsDavid Woodhouse
Mostly made redundant by using dev_name() instead of pci_name(), and one instance of using *dev->dma_mask instead of pdev->dma_mask. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make get_valid_domain_for_dev() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make iommu_should_identity_map() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Handle RMRRs for non-PCI devicesDavid Woodhouse
Should hopefully never happen (RMRRs are an abomination) but while we're busy eliminating all the PCI assumptions, we might as well do it. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make get_domain_for_dev() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make domain_context_mapp{ed,ing}() take struct deviceDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make device_to_iommu() cope with non-PCI devicesDavid Woodhouse
Pass the struct device to it, and also make it return the bus/devfn to use, since that is also stored in the DMAR table. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make identity_mapping() take struct device not struct pci_devDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove segment from struct device_domain_info()David Woodhouse
It's accessible via info->iommu->segment so this is redundant. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Remove device_to_iommu() call from domain_remove_dev_info()David Woodhouse
This was problematic because it works by domain/bus/devfn and we want to make device_to_iommu() use only a struct device * (for handling non-PCI devices). Now that the iommu pointer is reliably stored in the device_domain_info, we don't need to look it up. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Simplify iommu check in domain_remove_one_dev_info()David Woodhouse
Now we store the iommu in the device_domain_info, we don't need to do a lookup. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Always store iommu in device_domain_infoDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Use domain_remove_one_dev_info() in domain_add_dev_info() error pathDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: use dmar_insert_dev_info() from dma_add_dev_info()David Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Stop dmar_insert_dev_info() freeing domains on losing raceDavid Woodhouse
By moving this into get_domain_for_dev() we can make dmar_insert_dev_info() suitable for use with "special" domains such as the si_domain, which currently use domain_add_dev_info(). Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Pass iommu to domain_context_mapping_one() and ↵David Woodhouse
iommu_support_dev_iotlb() Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Use struct device in device_domain_info, not struct pci_devDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make dmar_insert_dev_info() take struct device instead of struct ↵David Woodhouse
pci_dev Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Make iommu_dummy() take struct device instead of struct pci_devDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24iommu/vt-d: Change scope lists to struct device, bus, devfnDavid Woodhouse
It's not only for PCI devices any more, and the scope information for an ACPI device provides the bus and devfn so that has to be stored here too. It is the device pointer itself which needs to be protected with RCU, so the __rcu annotation follows it into the definition of struct dmar_dev_scope, since we're no longer just passing arrays of device pointers around. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19iommu/vt-d: Be less pessimistic about domain coherency where possibleDavid Woodhouse
In commit 2e12bc29 ("intel-iommu: Default to non-coherent for domains unattached to iommus") we decided to err on the side of caution and always assume that it's possible that a device will be attached which is behind a non-coherent IOMMU. In some cases, however, that just *cannot* happen. If there *are* no IOMMUs in the system which are non-coherent, then we don't need to do it. And flushing the dcache is a *significant* performance hit. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19iommu/vt-d: Honour intel_iommu=sp_off for non-VMM domainsDavid Woodhouse
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19iommu/vt-d: Clean up and fix page table clear/free behaviourDavid Woodhouse
There is a race condition between the existing clear/free code and the hardware. The IOMMU is actually permitted to cache the intermediate levels of the page tables, and doesn't need to walk the table from the very top of the PGD each time. So the existing back-to-back calls to dma_pte_clear_range() and dma_pte_free_pagetable() can lead to a use-after-free where the IOMMU reads from a freed page table. When freeing page tables we actually need to do the IOTLB flush, with the 'invalidation hint' bit clear to indicate that it's not just a leaf-node flush, after unlinking each page table page from the next level up but before actually freeing it. So in the rewritten domain_unmap() we just return a list of pages (using pg->freelist to make a list of them), and then the caller is expected to do the appropriate IOTLB flush (or tear down the domain completely, whatever), before finally calling dma_free_pagelist() to free the pages. As an added bonus, we no longer need to flush the CPU's data cache for pages which are about to be *removed* from the page table hierarchy anyway, in the non-cache-coherent case. This drastically improves the performance of large unmaps. As a side-effect of all these changes, this also fixes the fact that intel_iommu_unmap() was neglecting to free the page tables for the range in question after clearing them. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19iommu/vt-d: Clean up size handling for intel_iommu_unmap()David Woodhouse
We have this horrid API where iommu_unmap() can unmap more than it's asked to, if the IOVA in question happens to be mapped with a large page. Instead of propagating this nonsense to the point where we end up returning the page order from dma_pte_clear_range(), let's just do it once and adjust the 'size' parameter accordingly. Augment pfn_to_dma_pte() to return the level at which the PTE was found, which will also be useful later if we end up changing the API for iommu_iova_to_phys() to behave the same way as is being discussed upstream. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-04iommu/vt-d: Update IOMMU state when memory hotplug happensJiang Liu
If static identity domain is created, IOMMU driver needs to update si_domain page table when memory hotplug event happens. Otherwise PCI device DMA operations can't access the hot-added memory regions. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04iommu/vt-d: Unify the way to process DMAR device scope arrayJiang Liu
Now we have a PCI bus notification based mechanism to update DMAR device scope array, we could extend the mechanism to support boot time initialization too, which will help to unify and simplify the implementation. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04iommu/vt-d: Update DRHD/RMRR/ATSR device scope caches when PCI hotplug happensJiang Liu
Current Intel DMAR/IOMMU driver assumes that all PCI devices associated with DMAR/RMRR/ATSR device scope arrays are created at boot time and won't change at runtime, so it caches pointers of associated PCI device object. That assumption may be wrong now due to: 1) introduction of PCI host bridge hotplug 2) PCI device hotplug through sysfs interfaces. Wang Yijing has tried to solve this issue by caching <bus, dev, func> tupple instead of the PCI device object pointer, but that's still unreliable because PCI bus number may change in case of hotplug. Please refer to http://lkml.org/lkml/2013/11/5/64 Message from Yingjing's mail: after remove and rescan a pci device [ 611.857095] dmar: DRHD: handling fault status reg 2 [ 611.857109] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff7000 [ 611.857109] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.857524] dmar: DRHD: handling fault status reg 102 [ 611.857534] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff6000 [ 611.857534] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.857936] dmar: DRHD: handling fault status reg 202 [ 611.857947] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff5000 [ 611.857947] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.858351] dmar: DRHD: handling fault status reg 302 [ 611.858362] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff4000 [ 611.858362] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.860819] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready [ 611.860983] dmar: DRHD: handling fault status reg 402 [ 611.860995] dmar: INTR-REMAP: Request device [[86:00.3] fault index a4 [ 611.860995] INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear This patch introduces a new mechanism to update the DRHD/RMRR/ATSR device scope caches by hooking PCI bus notification. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04iommu/vt-d: Use RCU to protect global resources in interrupt contextJiang Liu
Global DMA and interrupt remapping resources may be accessed in interrupt context, so use RCU instead of rwsem to protect them in such cases. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04iommu/vt-d: Introduce a rwsem to protect global data structuresJiang Liu
Introduce a global rwsem dmar_global_lock, which will be used to protect DMAR related global data structures from DMAR/PCI/memory device hotplug operations in process context. DMA and interrupt remapping related data structures are read most, and only change when memory/PCI/DMAR hotplug event happens. So a global rwsem solution is adopted for balance between simplicity and performance. For interrupt remapping driver, function intel_irq_remapping_supported(), dmar_table_init(), intel_enable_irq_remapping(), disable_irq_remapping(), reenable_irq_remapping() and enable_drhd_fault_handling() etc are called during booting, suspending and resuming with interrupt disabled, so no need to take the global lock. For interrupt remapping entry allocation, the locking model is: down_read(&dmar_global_lock); /* Find corresponding iommu */ iommu = map_hpet_to_ir(id); if (iommu) /* * Allocate remapping entry and mark entry busy, * the IOMMU won't be hot-removed until the * allocated entry has been released. */ index = alloc_irte(iommu, irq, 1); up_read(&dmar_global_lock); For DMA remmaping driver, we only uses the dmar_global_lock rwsem to protect functions which are only called in process context. For any function which may be called in interrupt context, we will use RCU to protect them in following patches. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
generated by cgit (git 2.34.1) at 2025-07-18 15:06:03 +0000