* swiotlb/virtio: unchecked device dma address and length @ 2020-12-11 17:31 Felicitas Hetzelt 2020-12-14 21:49 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 27+ messages in thread From: Felicitas Hetzelt @ 2020-12-11 17:31 UTC (permalink / raw) To: virtualization, iommu Cc: Radev, Martin, david.kaplan, Michael S. Tsirkin, Jason Wang, konrad.wilk, Robert Buhren, Morbitzer, Mathias Hello, we have been analyzing the Hypervisor-OS interface of Linux and discovered bugs in the swiotlb/virtio implementation that can be triggered from a malicious Hypervisor / virtual device. With SEV, the SWIOTLB implementation is forcefully enabled and would always be used. Thus, all virtio devices and others would use it under the hood. The reason for analyzing this interface is that, technologies such as Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] change the threat model assumed by various Linux kernel subsystems. These technologies take the presence of a fully malicious hypervisor into account and aim to provide protection for virtual machines in such an environment. Therefore, all input received from the hypervisor or an external device should be carefully validated. Note that these issues are of little (or no) relevance in a "normal" virtualization setup, nevertheless we believe that it is required to fix them if TDX or SNP is used. We are happy to provide more information if needed! [1] https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html [2] https://www.amd.com/en/processors/amd-secure-encrypted-virtualization Bug: OOB memory write. dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr and length parameters that are under control of the device. This happens e.g. in virtio_ring: https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 This raises two issues: 1) swiotlb_tlb_unmap_single fails to check whether the index generated from the dma_addr is in range of the io_tlb_orig_addr array. 2) when swiotlb_bounce is called the device controls the length of the memory copied to the cpu address. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-11 17:31 swiotlb/virtio: unchecked device dma address and length Felicitas Hetzelt @ 2020-12-14 21:49 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-14 21:49 UTC (permalink / raw) To: Felicitas Hetzelt, ashish.kalra, jun.nakajima, hch Cc: Radev, Martin, david.kaplan, Michael S. Tsirkin, Jason Wang, virtualization, Robert Buhren, iommu, Morbitzer, Mathias On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > Hello, Hi! Please see below my responses. > we have been analyzing the Hypervisor-OS interface of Linux > and discovered bugs in the swiotlb/virtio implementation that can be > triggered from a malicious Hypervisor / virtual device. > With SEV, the SWIOTLB implementation is forcefully enabled and would > always be used. Thus, all virtio devices and others would use it under > the hood. > > The reason for analyzing this interface is that, technologies such as > Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > change the threat model assumed by various Linux kernel subsystems. > These technologies take the presence of a fully malicious hypervisor > into account and aim to provide protection for virtual machines in such > an environment. Therefore, all input received from the hypervisor or an > external device should be carefully validated. Note that these issues > are of little (or no) relevance in a "normal" virtualization setup, > nevertheless we believe that it is required to fix them if TDX or SNP is > used. > > We are happy to provide more information if needed! > > [1] > https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html > > [2] https://www.amd.com/en/processors/amd-secure-encrypted-virtualization > > Bug: > OOB memory write. > dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > and length parameters that are under control of the device. > This happens e.g. in virtio_ring: > https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 Heya! Thank you for pointing this out! I've a couple of questions and hope you can help me out with them. Also CC-ing AMD / TDX folks. > > This raises two issues: > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > from the dma_addr is in range of the io_tlb_orig_addr array. That is fairly simple to implement I would think. That is it can check that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force is used. > 2) when swiotlb_bounce is called the device controls the length of the > memory copied to the cpu address. So.. this sounds very similar to the Intel Thunder.. something issue where this exact issue was fixed by handing the DMA off to the SWIOTLB bounce code. But if that is broken, then that CVE is still not fixed? So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is under the attacker control. Ugh. One way could be to have a io_tlb_orig_addr-ish array with the length of mappings to double check? Couple more questions: - Did you have already some PoC fixes for this? - Is there a CVE associated with this? - Is there a paper on this you all are working on? Thank you! _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-14 21:49 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-14 21:49 UTC (permalink / raw) To: Felicitas Hetzelt, ashish.kalra, jun.nakajima, hch Cc: Radev, Martin, david.kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Morbitzer, Mathias On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > Hello, Hi! Please see below my responses. > we have been analyzing the Hypervisor-OS interface of Linux > and discovered bugs in the swiotlb/virtio implementation that can be > triggered from a malicious Hypervisor / virtual device. > With SEV, the SWIOTLB implementation is forcefully enabled and would > always be used. Thus, all virtio devices and others would use it under > the hood. > > The reason for analyzing this interface is that, technologies such as > Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > change the threat model assumed by various Linux kernel subsystems. > These technologies take the presence of a fully malicious hypervisor > into account and aim to provide protection for virtual machines in such > an environment. Therefore, all input received from the hypervisor or an > external device should be carefully validated. Note that these issues > are of little (or no) relevance in a "normal" virtualization setup, > nevertheless we believe that it is required to fix them if TDX or SNP is > used. > > We are happy to provide more information if needed! > > [1] > https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html > > [2] https://www.amd.com/en/processors/amd-secure-encrypted-virtualization > > Bug: > OOB memory write. > dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > and length parameters that are under control of the device. > This happens e.g. in virtio_ring: > https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 Heya! Thank you for pointing this out! I've a couple of questions and hope you can help me out with them. Also CC-ing AMD / TDX folks. > > This raises two issues: > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > from the dma_addr is in range of the io_tlb_orig_addr array. That is fairly simple to implement I would think. That is it can check that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force is used. > 2) when swiotlb_bounce is called the device controls the length of the > memory copied to the cpu address. So.. this sounds very similar to the Intel Thunder.. something issue where this exact issue was fixed by handing the DMA off to the SWIOTLB bounce code. But if that is broken, then that CVE is still not fixed? So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is under the attacker control. Ugh. One way could be to have a io_tlb_orig_addr-ish array with the length of mappings to double check? Couple more questions: - Did you have already some PoC fixes for this? - Is there a CVE associated with this? - Is there a paper on this you all are working on? Thank you! _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-14 21:49 ` Konrad Rzeszutek Wilk @ 2020-12-15 3:20 ` Jason Wang -1 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-15 3:20 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Felicitas Hetzelt, ashish.kalra, jun.nakajima, hch Cc: Radev, Martin, david.kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Morbitzer, Mathias On 2020/12/15 上午5:49, Konrad Rzeszutek Wilk wrote: > On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: >> Hello, > Hi! Please see below my responses. > >> we have been analyzing the Hypervisor-OS interface of Linux >> and discovered bugs in the swiotlb/virtio implementation that can be >> triggered from a malicious Hypervisor / virtual device. >> With SEV, the SWIOTLB implementation is forcefully enabled and would >> always be used. Thus, all virtio devices and others would use it under >> the hood. >> >> The reason for analyzing this interface is that, technologies such as >> Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] >> change the threat model assumed by various Linux kernel subsystems. >> These technologies take the presence of a fully malicious hypervisor >> into account and aim to provide protection for virtual machines in such >> an environment. Therefore, all input received from the hypervisor or an >> external device should be carefully validated. Note that these issues >> are of little (or no) relevance in a "normal" virtualization setup, >> nevertheless we believe that it is required to fix them if TDX or SNP is >> used. >> >> We are happy to provide more information if needed! >> >> [1] >> https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html >> >> [2]https://www.amd.com/en/processors/amd-secure-encrypted-virtualization >> >> Bug: >> OOB memory write. >> dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr >> and length parameters that are under control of the device. >> This happens e.g. in virtio_ring: >> https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 > Heya! > > Thank you for pointing this out! I've a couple of questions and hope you can > help me out with them. > > Also CC-ing AMD / TDX folks. >> This raises two issues: >> 1) swiotlb_tlb_unmap_single fails to check whether the index generated >> from the dma_addr is in range of the io_tlb_orig_addr array. > That is fairly simple to implement I would think. That is it can check > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > is used. I'm not sure this can fix all the cases. It looks to me we should map descriptor coherent but readonly (which is not supported by current DMA API). Otherwise, device can modify the desc[i].addr/desc[i].len at any time to pretend a valid mapping. Thanks > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-15 3:20 ` Jason Wang 0 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-15 3:20 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Felicitas Hetzelt, ashish.kalra, jun.nakajima, hch Cc: Radev, Martin, david.kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Morbitzer, Mathias On 2020/12/15 上午5:49, Konrad Rzeszutek Wilk wrote: > On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: >> Hello, > Hi! Please see below my responses. > >> we have been analyzing the Hypervisor-OS interface of Linux >> and discovered bugs in the swiotlb/virtio implementation that can be >> triggered from a malicious Hypervisor / virtual device. >> With SEV, the SWIOTLB implementation is forcefully enabled and would >> always be used. Thus, all virtio devices and others would use it under >> the hood. >> >> The reason for analyzing this interface is that, technologies such as >> Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] >> change the threat model assumed by various Linux kernel subsystems. >> These technologies take the presence of a fully malicious hypervisor >> into account and aim to provide protection for virtual machines in such >> an environment. Therefore, all input received from the hypervisor or an >> external device should be carefully validated. Note that these issues >> are of little (or no) relevance in a "normal" virtualization setup, >> nevertheless we believe that it is required to fix them if TDX or SNP is >> used. >> >> We are happy to provide more information if needed! >> >> [1] >> https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html >> >> [2]https://www.amd.com/en/processors/amd-secure-encrypted-virtualization >> >> Bug: >> OOB memory write. >> dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr >> and length parameters that are under control of the device. >> This happens e.g. in virtio_ring: >> https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 > Heya! > > Thank you for pointing this out! I've a couple of questions and hope you can > help me out with them. > > Also CC-ing AMD / TDX folks. >> This raises two issues: >> 1) swiotlb_tlb_unmap_single fails to check whether the index generated >> from the dma_addr is in range of the io_tlb_orig_addr array. > That is fairly simple to implement I would think. That is it can check > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > is used. I'm not sure this can fix all the cases. It looks to me we should map descriptor coherent but readonly (which is not supported by current DMA API). Otherwise, device can modify the desc[i].addr/desc[i].len at any time to pretend a valid mapping. Thanks > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-15 3:20 ` Jason Wang @ 2020-12-15 14:27 ` Konrad Rzeszutek Wilk -1 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-15 14:27 UTC (permalink / raw) To: Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch .snip. > > > This raises two issues: > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > That is fairly simple to implement I would think. That is it can check > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > is used. > > > I'm not sure this can fix all the cases. It looks to me we should map > descriptor coherent but readonly (which is not supported by current DMA > API). I think I am missing something obvious here. The attacker is the hypervisor, aka the owner of the VirtIO device (ring0). The attacker is the one that provides the addr/len - having that readonly from a guest perspective does not change the fact that the hypervisor can modify the memory range by mapping it via a different virtual address in the hypervisor? (aka aliasing it). > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > pretend a valid mapping. With the swiotlb=force as long as addr/len are within the PA boundaries within the SWIOTLB pool this should be OK? After all that whole area is in cleartext and visible to the attacker. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-15 14:27 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-15 14:27 UTC (permalink / raw) To: Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch .snip. > > > This raises two issues: > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > That is fairly simple to implement I would think. That is it can check > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > is used. > > > I'm not sure this can fix all the cases. It looks to me we should map > descriptor coherent but readonly (which is not supported by current DMA > API). I think I am missing something obvious here. The attacker is the hypervisor, aka the owner of the VirtIO device (ring0). The attacker is the one that provides the addr/len - having that readonly from a guest perspective does not change the fact that the hypervisor can modify the memory range by mapping it via a different virtual address in the hypervisor? (aka aliasing it). > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > pretend a valid mapping. With the swiotlb=force as long as addr/len are within the PA boundaries within the SWIOTLB pool this should be OK? After all that whole area is in cleartext and visible to the attacker. _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-15 14:27 ` Konrad Rzeszutek Wilk @ 2020-12-16 5:53 ` Jason Wang -1 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-16 5:53 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch ----- Original Message ----- > .snip. > > > > This raises two issues: > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > > That is fairly simple to implement I would think. That is it can check > > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > > is used. > > > > > > I'm not sure this can fix all the cases. It looks to me we should map > > descriptor coherent but readonly (which is not supported by current DMA > > API). > > I think I am missing something obvious here. The attacker is the hypervisor, > aka > the owner of the VirtIO device (ring0). The attacker is the one that > provides the addr/len - having that readonly from a guest perspective > does not change the fact that the hypervisor can modify the memory range > by mapping it via a different virtual address in the hypervisor? (aka > aliasing it). Right, but if we allow hypervisor to provide arbitrary addr/len, does it mean hypervisor can read uncrypted content of encrypted memory of guest through swiotlb? Thanks > > > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > > pretend a valid mapping. > > With the swiotlb=force as long as addr/len are within the PA boundaries > within the SWIOTLB pool this should be OK? > > After all that whole area is in cleartext and visible to the attacker. > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-16 5:53 ` Jason Wang 0 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-16 5:53 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch ----- Original Message ----- > .snip. > > > > This raises two issues: > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > > That is fairly simple to implement I would think. That is it can check > > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > > is used. > > > > > > I'm not sure this can fix all the cases. It looks to me we should map > > descriptor coherent but readonly (which is not supported by current DMA > > API). > > I think I am missing something obvious here. The attacker is the hypervisor, > aka > the owner of the VirtIO device (ring0). The attacker is the one that > provides the addr/len - having that readonly from a guest perspective > does not change the fact that the hypervisor can modify the memory range > by mapping it via a different virtual address in the hypervisor? (aka > aliasing it). Right, but if we allow hypervisor to provide arbitrary addr/len, does it mean hypervisor can read uncrypted content of encrypted memory of guest through swiotlb? Thanks > > > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > > pretend a valid mapping. > > With the swiotlb=force as long as addr/len are within the PA boundaries > within the SWIOTLB pool this should be OK? > > After all that whole area is in cleartext and visible to the attacker. > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 5:53 ` Jason Wang @ 2020-12-16 6:41 ` Jason Wang -1 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-16 6:41 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch ----- Original Message ----- > > > ----- Original Message ----- > > .snip. > > > > > This raises two issues: > > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index > > > > > generated > > > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > > > That is fairly simple to implement I would think. That is it can check > > > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > > > is used. > > > > > > > > > I'm not sure this can fix all the cases. It looks to me we should map > > > descriptor coherent but readonly (which is not supported by current DMA > > > API). > > > > I think I am missing something obvious here. The attacker is the > > hypervisor, > > aka > > the owner of the VirtIO device (ring0). The attacker is the one that > > provides the addr/len - having that readonly from a guest perspective > > does not change the fact that the hypervisor can modify the memory range > > by mapping it via a different virtual address in the hypervisor? (aka > > aliasing it). > > Right, but if we allow hypervisor to provide arbitrary addr/len, does > it mean hypervisor can read encrypted content of encrypted memory of > guest through swiotlb? > > Thanks Actually not. I think you're right. Thanks > > > > > > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > > > pretend a valid mapping. > > > > With the swiotlb=force as long as addr/len are within the PA boundaries > > within the SWIOTLB pool this should be OK? > > > > After all that whole area is in cleartext and visible to the attacker. > > > > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-16 6:41 ` Jason Wang 0 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-16 6:41 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch ----- Original Message ----- > > > ----- Original Message ----- > > .snip. > > > > > This raises two issues: > > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index > > > > > generated > > > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > > > That is fairly simple to implement I would think. That is it can check > > > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > > > is used. > > > > > > > > > I'm not sure this can fix all the cases. It looks to me we should map > > > descriptor coherent but readonly (which is not supported by current DMA > > > API). > > > > I think I am missing something obvious here. The attacker is the > > hypervisor, > > aka > > the owner of the VirtIO device (ring0). The attacker is the one that > > provides the addr/len - having that readonly from a guest perspective > > does not change the fact that the hypervisor can modify the memory range > > by mapping it via a different virtual address in the hypervisor? (aka > > aliasing it). > > Right, but if we allow hypervisor to provide arbitrary addr/len, does > it mean hypervisor can read encrypted content of encrypted memory of > guest through swiotlb? > > Thanks Actually not. I think you're right. Thanks > > > > > > > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > > > pretend a valid mapping. > > > > With the swiotlb=force as long as addr/len are within the PA boundaries > > within the SWIOTLB pool this should be OK? > > > > After all that whole area is in cleartext and visible to the attacker. > > > > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 6:41 ` Jason Wang @ 2020-12-16 13:04 ` Konrad Rzeszutek Wilk -1 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-16 13:04 UTC (permalink / raw) To: Jason Wang Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch On December 16, 2020 1:41:48 AM EST, Jason Wang <jasowang@redhat.com> wrote: > > >----- Original Message ----- >> >> >> ----- Original Message ----- >> > .snip. >> > > > > This raises two issues: >> > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index >> > > > > generated >> > > > > from the dma_addr is in range of the io_tlb_orig_addr array. >> > > > That is fairly simple to implement I would think. That is it >can check >> > > > that the dma_addr is from the PA in the io_tlb pool when >SWIOTLB=force >> > > > is used. >> > > >> > > >> > > I'm not sure this can fix all the cases. It looks to me we should >map >> > > descriptor coherent but readonly (which is not supported by >current DMA >> > > API). >> > >> > I think I am missing something obvious here. The attacker is the >> > hypervisor, >> > aka >> > the owner of the VirtIO device (ring0). The attacker is the one >that >> > provides the addr/len - having that readonly from a guest >perspective >> > does not change the fact that the hypervisor can modify the memory >range >> > by mapping it via a different virtual address in the hypervisor? >(aka >> > aliasing it). >> >> Right, but if we allow hypervisor to provide arbitrary addr/len, does >> it mean hypervisor can read encrypted content of encrypted memory of >> guest through swiotlb? Yes . >> >> Thanks > >Actually not. I think you're right. Your sentence is very confusing. On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing the correct DMA address and length which SWIOTLB uses to match to its associated original PA address. Think original PA having a mapping to a PA in the SWIOTLB pool. The length is not checked so the attacker can modify that to say a huge number and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the SWIOTLB PA pool. > >Thanks > >> >> > > >> > > Otherwise, device can modify the desc[i].addr/desc[i].len at any >time to >> > > pretend a valid mapping. >> > >> > With the swiotlb=force as long as addr/len are within the PA >boundaries >> > within the SWIOTLB pool this should be OK? >> > >> > After all that whole area is in cleartext and visible to the >attacker. >> > >> > >> _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-16 13:04 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-16 13:04 UTC (permalink / raw) To: Jason Wang Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch On December 16, 2020 1:41:48 AM EST, Jason Wang <jasowang@redhat.com> wrote: > > >----- Original Message ----- >> >> >> ----- Original Message ----- >> > .snip. >> > > > > This raises two issues: >> > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index >> > > > > generated >> > > > > from the dma_addr is in range of the io_tlb_orig_addr array. >> > > > That is fairly simple to implement I would think. That is it >can check >> > > > that the dma_addr is from the PA in the io_tlb pool when >SWIOTLB=force >> > > > is used. >> > > >> > > >> > > I'm not sure this can fix all the cases. It looks to me we should >map >> > > descriptor coherent but readonly (which is not supported by >current DMA >> > > API). >> > >> > I think I am missing something obvious here. The attacker is the >> > hypervisor, >> > aka >> > the owner of the VirtIO device (ring0). The attacker is the one >that >> > provides the addr/len - having that readonly from a guest >perspective >> > does not change the fact that the hypervisor can modify the memory >range >> > by mapping it via a different virtual address in the hypervisor? >(aka >> > aliasing it). >> >> Right, but if we allow hypervisor to provide arbitrary addr/len, does >> it mean hypervisor can read encrypted content of encrypted memory of >> guest through swiotlb? Yes . >> >> Thanks > >Actually not. I think you're right. Your sentence is very confusing. On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing the correct DMA address and length which SWIOTLB uses to match to its associated original PA address. Think original PA having a mapping to a PA in the SWIOTLB pool. The length is not checked so the attacker can modify that to say a huge number and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the SWIOTLB PA pool. > >Thanks > >> >> > > >> > > Otherwise, device can modify the desc[i].addr/desc[i].len at any >time to >> > > pretend a valid mapping. >> > >> > With the swiotlb=force as long as addr/len are within the PA >boundaries >> > within the SWIOTLB pool this should be OK? >> > >> > After all that whole area is in cleartext and visible to the >attacker. >> > >> > >> _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 13:04 ` Konrad Rzeszutek Wilk @ 2020-12-17 4:19 ` Jason Wang -1 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-17 4:19 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch On 2020/12/16 下午9:04, Konrad Rzeszutek Wilk wrote: > On December 16, 2020 1:41:48 AM EST, Jason Wang <jasowang@redhat.com> wrote: >> >> ----- Original Message ----- >>> >>> ----- Original Message ----- >>>> .snip. >>>>>>> This raises two issues: >>>>>>> 1) swiotlb_tlb_unmap_single fails to check whether the index >>>>>>> generated >>>>>>> from the dma_addr is in range of the io_tlb_orig_addr array. >>>>>> That is fairly simple to implement I would think. That is it >> can check >>>>>> that the dma_addr is from the PA in the io_tlb pool when >> SWIOTLB=force >>>>>> is used. >>>>> >>>>> I'm not sure this can fix all the cases. It looks to me we should >> map >>>>> descriptor coherent but readonly (which is not supported by >> current DMA >>>>> API). >>>> I think I am missing something obvious here. The attacker is the >>>> hypervisor, >>>> aka >>>> the owner of the VirtIO device (ring0). The attacker is the one >> that >>>> provides the addr/len - having that readonly from a guest >> perspective >>>> does not change the fact that the hypervisor can modify the memory >> range >>>> by mapping it via a different virtual address in the hypervisor? >> (aka >>>> aliasing it). >>> Right, but if we allow hypervisor to provide arbitrary addr/len, does >>> it mean hypervisor can read encrypted content of encrypted memory of >>> guest through swiotlb? > Yes . >>> Thanks >> Actually not. I think you're right. > > Your sentence is very confusing. Sorry for being unclear. This is all a reply to your suggestion of adding checks in the swiotlb. > > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing the correct DMA address and length which SWIOTLB uses to match to its associated original PA address. > > Think original PA having a mapping to a PA in the SWIOTLB pool. > > > The length is not checked so the attacker can modify that to say a huge number and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the SWIOTLB PA pool. How can we read in this case? It looks to me we don't try to read during dma_unmap(). Thanks > > > > >> Thanks >> >>>>> Otherwise, device can modify the desc[i].addr/desc[i].len at any >> time to >>>>> pretend a valid mapping. >>>> With the swiotlb=force as long as addr/len are within the PA >> boundaries >>>> within the SWIOTLB pool this should be OK? >>>> >>>> After all that whole area is in cleartext and visible to the >> attacker. >>>> _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-17 4:19 ` Jason Wang 0 siblings, 0 replies; 27+ messages in thread From: Jason Wang @ 2020-12-17 4:19 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: ashish kalra, Felicitas Hetzelt, Martin Radev, david kaplan, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, Mathias Morbitzer, hch On 2020/12/16 下午9:04, Konrad Rzeszutek Wilk wrote: > On December 16, 2020 1:41:48 AM EST, Jason Wang <jasowang@redhat.com> wrote: >> >> ----- Original Message ----- >>> >>> ----- Original Message ----- >>>> .snip. >>>>>>> This raises two issues: >>>>>>> 1) swiotlb_tlb_unmap_single fails to check whether the index >>>>>>> generated >>>>>>> from the dma_addr is in range of the io_tlb_orig_addr array. >>>>>> That is fairly simple to implement I would think. That is it >> can check >>>>>> that the dma_addr is from the PA in the io_tlb pool when >> SWIOTLB=force >>>>>> is used. >>>>> >>>>> I'm not sure this can fix all the cases. It looks to me we should >> map >>>>> descriptor coherent but readonly (which is not supported by >> current DMA >>>>> API). >>>> I think I am missing something obvious here. The attacker is the >>>> hypervisor, >>>> aka >>>> the owner of the VirtIO device (ring0). The attacker is the one >> that >>>> provides the addr/len - having that readonly from a guest >> perspective >>>> does not change the fact that the hypervisor can modify the memory >> range >>>> by mapping it via a different virtual address in the hypervisor? >> (aka >>>> aliasing it). >>> Right, but if we allow hypervisor to provide arbitrary addr/len, does >>> it mean hypervisor can read encrypted content of encrypted memory of >>> guest through swiotlb? > Yes . >>> Thanks >> Actually not. I think you're right. > > Your sentence is very confusing. Sorry for being unclear. This is all a reply to your suggestion of adding checks in the swiotlb. > > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing the correct DMA address and length which SWIOTLB uses to match to its associated original PA address. > > Think original PA having a mapping to a PA in the SWIOTLB pool. > > > The length is not checked so the attacker can modify that to say a huge number and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the SWIOTLB PA pool. How can we read in this case? It looks to me we don't try to read during dma_unmap(). Thanks > > > > >> Thanks >> >>>>> Otherwise, device can modify the desc[i].addr/desc[i].len at any >> time to >>>>> pretend a valid mapping. >>>> With the swiotlb=force as long as addr/len are within the PA >> boundaries >>>> within the SWIOTLB pool this should be OK? >>>> >>>> After all that whole area is in cleartext and visible to the >> attacker. >>>> _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-17 4:19 ` Jason Wang (?) @ 2020-12-17 22:55 ` Ashish Kalra -1 siblings, 0 replies; 27+ messages in thread From: Ashish Kalra @ 2020-12-17 22:55 UTC (permalink / raw) To: Jason Wang Cc: Thomas.Lendacky, Jon.Grimm, Felicitas Hetzelt, Martin Radev, david kaplan, Konrad Rzeszutek Wilk, Michael S. Tsirkin, virtualization, Robert Buhren, iommu, brijesh.singh, Mathias Morbitzer, hch On Thu, Dec 17, 2020 at 12:19:16PM +0800, Jason Wang wrote: > > On 2020/12/16 下午9:04, Konrad Rzeszutek Wilk wrote: > > On December 16, 2020 1:41:48 AM EST, Jason Wang <jasowang@redhat.com> wrote: > > > > > > ----- Original Message ----- > > > > > > > > ----- Original Message ----- > > > > > .snip. > > > > > > > > This raises two issues: > > > > > > > > 1) swiotlb_tlb_unmap_single fails to check whether the index > > > > > > > > generated > > > > > > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > > > > > > That is fairly simple to implement I would think. That is it > > > can check > > > > > > > that the dma_addr is from the PA in the io_tlb pool when > > > SWIOTLB=force > > > > > > > is used. > > > > > > > > > > > > I'm not sure this can fix all the cases. It looks to me we should > > > map > > > > > > descriptor coherent but readonly (which is not supported by > > > current DMA > > > > > > API). > > > > > I think I am missing something obvious here. The attacker is the > > > > > hypervisor, > > > > > aka > > > > > the owner of the VirtIO device (ring0). The attacker is the one > > > that > > > > > provides the addr/len - having that readonly from a guest > > > perspective > > > > > does not change the fact that the hypervisor can modify the memory > > > range > > > > > by mapping it via a different virtual address in the hypervisor? > > > (aka > > > > > aliasing it). > > > > Right, but if we allow hypervisor to provide arbitrary addr/len, does > > > > it mean hypervisor can read encrypted content of encrypted memory of > > > > guest through swiotlb? > > Yes . > > > > Thanks > > > Actually not. I think you're right. > > > > Your sentence is very confusing. > > > Sorry for being unclear. This is all a reply to your suggestion of adding > checks in the swiotlb. > > > > > > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing the correct DMA address and length which SWIOTLB uses to match to its associated original PA address. > > > > Think original PA having a mapping to a PA in the SWIOTLB pool. > > > > > > The length is not checked so the attacker can modify that to say a huge number and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the SWIOTLB PA pool. > > > How can we read in this case? It looks to me we don't try to read during > dma_unmap(). > That seems to be correct as in the unmap path, swiotlb_bounce() is being called with DMA_FROM_DEVICE flag, so there is no read involved during dma_unmap(). Thanks, Ashish > > > > > > > > > > > > > Thanks > > > > > > > > > Otherwise, device can modify the desc[i].addr/desc[i].len at any > > > time to > > > > > > pretend a valid mapping. > > > > > With the swiotlb=force as long as addr/len are within the PA > > > boundaries > > > > > within the SWIOTLB pool this should be OK? > > > > > > > > > > After all that whole area is in cleartext and visible to the > > > attacker. > > > > > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-15 3:20 ` Jason Wang @ 2020-12-16 8:54 ` Michael S. Tsirkin -1 siblings, 0 replies; 27+ messages in thread From: Michael S. Tsirkin @ 2020-12-16 8:54 UTC (permalink / raw) To: Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, Konrad Rzeszutek Wilk, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch On Tue, Dec 15, 2020 at 11:20:48AM +0800, Jason Wang wrote: > > On 2020/12/15 上午5:49, Konrad Rzeszutek Wilk wrote: > > On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > > > Hello, > > Hi! Please see below my responses. > > > > > we have been analyzing the Hypervisor-OS interface of Linux > > > and discovered bugs in the swiotlb/virtio implementation that can be > > > triggered from a malicious Hypervisor / virtual device. > > > With SEV, the SWIOTLB implementation is forcefully enabled and would > > > always be used. Thus, all virtio devices and others would use it under > > > the hood. > > > > > > The reason for analyzing this interface is that, technologies such as > > > Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > > > change the threat model assumed by various Linux kernel subsystems. > > > These technologies take the presence of a fully malicious hypervisor > > > into account and aim to provide protection for virtual machines in such > > > an environment. Therefore, all input received from the hypervisor or an > > > external device should be carefully validated. Note that these issues > > > are of little (or no) relevance in a "normal" virtualization setup, > > > nevertheless we believe that it is required to fix them if TDX or SNP is > > > used. > > > > > > We are happy to provide more information if needed! > > > > > > [1] > > > https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html > > > > > > [2]https://www.amd.com/en/processors/amd-secure-encrypted-virtualization > > > > > > Bug: > > > OOB memory write. > > > dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > > > and length parameters that are under control of the device. > > > This happens e.g. in virtio_ring: > > > https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 > > Heya! > > > > Thank you for pointing this out! I've a couple of questions and hope you can > > help me out with them. > > > > Also CC-ing AMD / TDX folks. > > > This raises two issues: > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > That is fairly simple to implement I would think. That is it can check > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > is used. > > > I'm not sure this can fix all the cases. It looks to me we should map > descriptor coherent but readonly (which is not supported by current DMA > API). Neither is this supported but encrypted memory technologies. > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > pretend a valid mapping. > > Thanks > > > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-16 8:54 ` Michael S. Tsirkin 0 siblings, 0 replies; 27+ messages in thread From: Michael S. Tsirkin @ 2020-12-16 8:54 UTC (permalink / raw) To: Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, Konrad Rzeszutek Wilk, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch On Tue, Dec 15, 2020 at 11:20:48AM +0800, Jason Wang wrote: > > On 2020/12/15 上午5:49, Konrad Rzeszutek Wilk wrote: > > On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > > > Hello, > > Hi! Please see below my responses. > > > > > we have been analyzing the Hypervisor-OS interface of Linux > > > and discovered bugs in the swiotlb/virtio implementation that can be > > > triggered from a malicious Hypervisor / virtual device. > > > With SEV, the SWIOTLB implementation is forcefully enabled and would > > > always be used. Thus, all virtio devices and others would use it under > > > the hood. > > > > > > The reason for analyzing this interface is that, technologies such as > > > Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > > > change the threat model assumed by various Linux kernel subsystems. > > > These technologies take the presence of a fully malicious hypervisor > > > into account and aim to provide protection for virtual machines in such > > > an environment. Therefore, all input received from the hypervisor or an > > > external device should be carefully validated. Note that these issues > > > are of little (or no) relevance in a "normal" virtualization setup, > > > nevertheless we believe that it is required to fix them if TDX or SNP is > > > used. > > > > > > We are happy to provide more information if needed! > > > > > > [1] > > > https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html > > > > > > [2]https://www.amd.com/en/processors/amd-secure-encrypted-virtualization > > > > > > Bug: > > > OOB memory write. > > > dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > > > and length parameters that are under control of the device. > > > This happens e.g. in virtio_ring: > > > https://elixir.bootlin.com/linux/v5.10-rc7/source/drivers/virtio/virtio_ring.c#L378 > > Heya! > > > > Thank you for pointing this out! I've a couple of questions and hope you can > > help me out with them. > > > > Also CC-ing AMD / TDX folks. > > > This raises two issues: > > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > > from the dma_addr is in range of the io_tlb_orig_addr array. > > That is fairly simple to implement I would think. That is it can check > > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > > is used. > > > I'm not sure this can fix all the cases. It looks to me we should map > descriptor coherent but readonly (which is not supported by current DMA > API). Neither is this supported but encrypted memory technologies. > Otherwise, device can modify the desc[i].addr/desc[i].len at any time to > pretend a valid mapping. > > Thanks > > > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 8:54 ` Michael S. Tsirkin @ 2020-12-16 13:07 ` Konrad Rzeszutek Wilk -1 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-16 13:07 UTC (permalink / raw) To: Michael S. Tsirkin, Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch ..snip.. >> > > This raises two issues: >> > > 1) swiotlb_tlb_unmap_single fails to check whether the index >generated >> > > from the dma_addr is in range of the io_tlb_orig_addr array. >> > That is fairly simple to implement I would think. That is it can >check >> > that the dma_addr is from the PA in the io_tlb pool when >SWIOTLB=force >> > is used. >> >> >> I'm not sure this can fix all the cases. It looks to me we should map >> descriptor coherent but readonly (which is not supported by current >DMA >> API). > >Neither is this supported but encrypted memory technologies. -ECONFUSED. Could you state this once more please? I am not exactly sure what you are saying > >> Otherwise, device can modify the desc[i].addr/desc[i].len at any time >to >> pretend a valid mapping. >> >> Thanks >> >> >> > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-16 13:07 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-16 13:07 UTC (permalink / raw) To: Michael S. Tsirkin, Jason Wang Cc: ashish.kalra, Felicitas Hetzelt, Radev, Martin, david.kaplan, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch ..snip.. >> > > This raises two issues: >> > > 1) swiotlb_tlb_unmap_single fails to check whether the index >generated >> > > from the dma_addr is in range of the io_tlb_orig_addr array. >> > That is fairly simple to implement I would think. That is it can >check >> > that the dma_addr is from the PA in the io_tlb pool when >SWIOTLB=force >> > is used. >> >> >> I'm not sure this can fix all the cases. It looks to me we should map >> descriptor coherent but readonly (which is not supported by current >DMA >> API). > >Neither is this supported but encrypted memory technologies. -ECONFUSED. Could you state this once more please? I am not exactly sure what you are saying > >> Otherwise, device can modify the desc[i].addr/desc[i].len at any time >to >> pretend a valid mapping. >> >> Thanks >> >> >> > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 13:07 ` Konrad Rzeszutek Wilk (?) @ 2020-12-16 22:07 ` Radev, Martin 2020-12-17 23:17 ` Ashish Kalra -1 siblings, 1 reply; 27+ messages in thread From: Radev, Martin @ 2020-12-16 22:07 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Michael S. Tsirkin, Jason Wang Cc: ashish.kalra@amd.com, Felicitas Hetzelt, david.kaplan@amd.com, virtualization@lists.linux-foundation.org, Robert Buhren, iommu@lists.linux-foundation.org, Morbitzer, Mathias, hch@lst.de [-- Attachment #1.1: Type: text/plain, Size: 6177 bytes --] Hello everybody, I will try help clarify some things. > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing > the correct DMA address and length which SWIOTLB uses to match to its associated > original PA address. > The length is not checked so the attacker can modify that to say a huge number > and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the > SWIOTLB PA pool. This is true. As an example, I attached to the QEMU process, set a BP to `virtqueue_split_fill` and modified the length field from 0x40 to 0x10000, and filled the corresponding buffer in the swiotlb region with As (0x41). Immediately after resuming execution, the kernel would crash: [ 122.154142] general protection fault, probably for non-canonical address 0x4141414141414141: 0000 [#1] PREEMPT SMP NOPTI [ 122.156088] CPU: 0 PID: 917 Comm: kworker/0:6 Kdump: loaded Tainted: G W E 5.6.12-sevault+ #28 [ 122.157855] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 [ 122.159079] Workqueue: events_freezable_power_ disk_events_workfn [ 122.160040] RIP: 0010:scsi_queue_rq+0x5af/0xa70 [scsi_mod] [ 122.160916] Code: 01 89 83 9c 02 00 00 41 80 7f 08 00 74 07 83 8b 9c 02 00 00 08 48 8b 83 40 02 00 00 c7 83 3c 01 00 00 00 00 00 00 48 8d 78 08 <48> c7 00 00 00 00 00 48 c7 40 58 00 00 00 00 48 83 e7 f8 48 29 f8 [ 122.163821] RSP: 0018:ffffc900002efb08 EFLAGS: 00010202 [ 122.164637] RAX: 4141414141414141 RBX: ffff888035b89c00 RCX: ffff888035b89ed0 [ 122.165775] RDX: 0000000000000008 RSI: 0000000000000000 RDI: 4141414141414149 [ 122.166891] RBP: ffff888035946000 R08: ffff888035a79860 R09: 0000000000000000 [ 122.168016] R10: ffffea0001287280 R11: 0000000000000008 R12: ffff888035b89d18 [ 122.169159] R13: ffff888035945000 R14: ffff888035946000 R15: ffffc900002efba0 [ 122.170287] FS: 0000000000000000(0000) GS:ffff88807f800000(0000) knlGS:0000000000000000 [ 122.171564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 122.172470] CR2: 0000560e654b77b8 CR3: 000000004dd38000 CR4: 00000000003406f0 What and where gets overwritten entirely depends on what virtio driver is being targeted. All manage their memory for the descriptor buffers differently so the overwrite may require to be large. In the context of VirtIO and SWIOTLB, there are also these three fields other than the length: dma_addr, flags, next I had a look around a little bit, so my take is the following: 1) There's already validation for dma_addr before doing the unmap with a call to is_swiotlb_buffer (1). I think this check is sufficient. 2) flags Before doing the unmap, the virtio implementation would check the flag and based on it would select a DMA direction (TO/FROM DEVICE). Still, it seems that this would not trick the driver to copy data to the device since only a `sync for CPU` may be performed in the unmap path. However, I can see that this would eliminate a memcpy to any passed buffer which means that an uninitialized buffer may be passed to some client like a user space application. I tested that with the device `virtio-rng` which would leak some stored kernel address. 3) next I think this was reported by Felicitas that this can lead to an OOB access, but it is limited. So, I think that the problem of overwriting the addr and flags fields are not a big issue on their own. But there should be some validation for the size and that should be probably done the SWIOTLB implementation level. There's already an array to keep the original address (3), What about adding an extra array to keep track of also the `original size`. It would be populated when some memory is mapped, just like with io_tlb_orig_addr (4). Then the validation can be added in (5) and (6). This swiotlb/virtio issue affects the AMD SEV features where swiotlb is always forcefully enabled. I can also see that SWIOTLB is also always enabled for s390 but I don't know their threat model. [1] https://elixir.bootlin.com/linux/v5.10/source/include/linux/swiotlb.h#L72 [2] https://elixir.bootlin.com/linux/v5.8/source/drivers/virtio/virtio_ring.c#L381 [3] https://elixir.bootlin.com/linux/v5.8/source/kernel/dma/swiotlb.c#L103 [4] https://elixir.bootlin.com/linux/v5.8/source/kernel/dma/swiotlb.c#L570 [5] https://elixir.bootlin.com/linux/v5.8/source/kernel/dma/swiotlb.c#L588 [6] https://elixir.bootlin.com/linux/v5.8/source/kernel/dma/swiotlb.c#L633 ________________________________ From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Sent: Wednesday, December 16, 2020 2:07 PM To: Michael S. Tsirkin <mst@redhat.com>; Jason Wang <jasowang@redhat.com> Cc: Felicitas Hetzelt <file@sect.tu-berlin.de>; ashish.kalra@amd.com <ashish.kalra@amd.com>; jun.nakajima@intel.com <jun.nakajima@intel.com>; hch@lst.de <hch@lst.de>; virtualization@lists.linux-foundation.org <virtualization@lists.linux-foundation.org>; iommu@lists.linux-foundation.org <iommu@lists.linux-foundation.org>; Radev, Martin <martin.radev@aisec.fraunhofer.de>; Morbitzer, Mathias <mathias.morbitzer@aisec.fraunhofer.de>; Robert Buhren <robert@sect.tu-berlin.de>; david.kaplan@amd.com <david.kaplan@amd.com> Subject: Re: swiotlb/virtio: unchecked device dma address and length ..snip.. >> > > This raises two issues: >> > > 1) swiotlb_tlb_unmap_single fails to check whether the index >generated >> > > from the dma_addr is in range of the io_tlb_orig_addr array. >> > That is fairly simple to implement I would think. That is it can >check >> > that the dma_addr is from the PA in the io_tlb pool when >SWIOTLB=force >> > is used. >> >> >> I'm not sure this can fix all the cases. It looks to me we should map >> descriptor coherent but readonly (which is not supported by current >DMA >> API). > >Neither is this supported but encrypted memory technologies. -ECONFUSED. Could you state this once more please? I am not exactly sure what you are saying > >> Otherwise, device can modify the desc[i].addr/desc[i].len at any time >to >> pretend a valid mapping. >> >> Thanks >> >> >> > [-- Attachment #1.2: Type: text/html, Size: 11931 bytes --] [-- Attachment #2: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-16 22:07 ` Radev, Martin @ 2020-12-17 23:17 ` Ashish Kalra 2020-12-18 9:28 ` Radev, Martin 0 siblings, 1 reply; 27+ messages in thread From: Ashish Kalra @ 2020-12-17 23:17 UTC (permalink / raw) To: Radev, Martin Cc: Thomas.Lendacky, Jon.Grimm, Felicitas Hetzelt, david.kaplan@amd.com, Michael S. Tsirkin, Jason Wang, Konrad Rzeszutek Wilk, virtualization@lists.linux-foundation.org, Robert Buhren, iommu@lists.linux-foundation.org, brijesh.singh, Morbitzer, Mathias, hch@lst.de On Wed, Dec 16, 2020 at 10:07:31PM +0000, Radev, Martin wrote: > Hello everybody, > > I will try help clarify some things. > > > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing > > the correct DMA address and length which SWIOTLB uses to match to its associated > > original PA address. > > The length is not checked so the attacker can modify that to say a huge number > > and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the > > SWIOTLB PA pool. > > This is true. > As an example, I attached to the QEMU process, set a BP to `virtqueue_split_fill` > and modified the length field from 0x40 to 0x10000, and filled the corresponding > buffer in the swiotlb region with As (0x41). > > Immediately after resuming execution, the kernel would crash: > [ 122.154142] general protection fault, probably for non-canonical address 0x4141414141414141: 0000 [#1] PREEMPT SMP NOPTI > [ 122.156088] CPU: 0 PID: 917 Comm: kworker/0:6 Kdump: loaded Tainted: G W E 5.6.12-sevault+ #28 > [ 122.157855] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 > [ 122.159079] Workqueue: events_freezable_power_ disk_events_workfn > [ 122.160040] RIP: 0010:scsi_queue_rq+0x5af/0xa70 [scsi_mod] > [ 122.160916] Code: 01 89 83 9c 02 00 00 41 80 7f 08 00 74 07 83 8b 9c 02 00 00 08 48 8b 83 40 02 00 00 c7 83 3c 01 00 00 00 00 00 00 48 8d 78 08 <48> > c7 00 00 00 00 00 48 c7 40 58 00 00 00 00 48 83 e7 f8 48 29 f8 > [ 122.163821] RSP: 0018:ffffc900002efb08 EFLAGS: 00010202 > [ 122.164637] RAX: 4141414141414141 RBX: ffff888035b89c00 RCX: ffff888035b89ed0 > [ 122.165775] RDX: 0000000000000008 RSI: 0000000000000000 RDI: 4141414141414149 > [ 122.166891] RBP: ffff888035946000 R08: ffff888035a79860 R09: 0000000000000000 > [ 122.168016] R10: ffffea0001287280 R11: 0000000000000008 R12: ffff888035b89d18 > [ 122.169159] R13: ffff888035945000 R14: ffff888035946000 R15: ffffc900002efba0 > [ 122.170287] FS: 0000000000000000(0000) GS:ffff88807f800000(0000) knlGS:0000000000000000 > [ 122.171564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 122.172470] CR2: 0000560e654b77b8 CR3: 000000004dd38000 CR4: 00000000003406f0 > I believe the above example is without a SEV guest enabled/active, as SEV guest debugging can only be done with SEV Debug patches applied. > What and where gets overwritten entirely depends on what virtio driver is being > targeted. All manage their memory for the descriptor buffers differently so the overwrite > may require to be large. > > In the context of VirtIO and SWIOTLB, there are also these three fields other than the length: > dma_addr, flags, next > > I had a look around a little bit, so my take is the following: > > 1) There's already validation for dma_addr before doing the unmap with a call > to is_swiotlb_buffer (1). I think this check is sufficient. > > 2) flags > Before doing the unmap, the virtio implementation would check the flag and based on it > would select a DMA direction (TO/FROM DEVICE). Still, it seems that this would not > trick the driver to copy data to the device since only a `sync for CPU` may be performed > in the unmap path. That seems to be true. Thanks, Ashish _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-17 23:17 ` Ashish Kalra @ 2020-12-18 9:28 ` Radev, Martin 0 siblings, 0 replies; 27+ messages in thread From: Radev, Martin @ 2020-12-18 9:28 UTC (permalink / raw) To: Ashish Kalra Cc: Thomas.Lendacky@amd.com, Jon.Grimm@amd.com, Felicitas Hetzelt, david.kaplan@amd.com, Michael S. Tsirkin, Jason Wang, Konrad Rzeszutek Wilk, virtualization@lists.linux-foundation.org, Robert Buhren, iommu@lists.linux-foundation.org, brijesh.singh@amd.com, Morbitzer, Mathias, hch@lst.de [-- Attachment #1.1: Type: text/plain, Size: 4707 bytes --] > I believe the above example is without a SEV guest enabled/active, > as SEVguest debugging can only be done with SEV Debug patches applied. SEV is active. Adding `console=ttyS0` to the kernel parameters and `GRUB_TERMINAL="console serial"` seems to do the job without any AMD-internal patches. Note that even without SEV, I could get an exact repro by adding `swiotlb=force` to the kernel cmd and `iommu_platform=on` to the qemu virtio device. ________________________________ From: Ashish Kalra <ashish.kalra@amd.com> Sent: Friday, December 18, 2020 12:17 AM To: Radev, Martin <martin.radev@aisec.fraunhofer.de> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; Michael S. Tsirkin <mst@redhat.com>; Jason Wang <jasowang@redhat.com>; Felicitas Hetzelt <file@sect.tu-berlin.de>; jun.nakajima@intel.com <jun.nakajima@intel.com>; hch@lst.de <hch@lst.de>; virtualization@lists.linux-foundation.org <virtualization@lists.linux-foundation.org>; iommu@lists.linux-foundation.org <iommu@lists.linux-foundation.org>; Morbitzer, Mathias <mathias.morbitzer@aisec.fraunhofer.de>; Robert Buhren <robert@sect.tu-berlin.de>; david.kaplan@amd.com <david.kaplan@amd.com>; Thomas.Lendacky@amd.com <Thomas.Lendacky@amd.com>; Jon.Grimm@amd.com <Jon.Grimm@amd.com>; brijesh.singh@amd.com <brijesh.singh@amd.com> Subject: Re: swiotlb/virtio: unchecked device dma address and length On Wed, Dec 16, 2020 at 10:07:31PM +0000, Radev, Martin wrote: > Hello everybody, > > I will try help clarify some things. > > > On a DMA unmap SWIOTLB (when force is used) it trusts the driver from providing > > the correct DMA address and length which SWIOTLB uses to match to its associated > > original PA address. > > The length is not checked so the attacker can modify that to say a huge number > > and cause SWIOTLB bounce code to write or read data from non SWIOTLB PA into the > > SWIOTLB PA pool. > > This is true. > As an example, I attached to the QEMU process, set a BP to `virtqueue_split_fill` > and modified the length field from 0x40 to 0x10000, and filled the corresponding > buffer in the swiotlb region with As (0x41). > > Immediately after resuming execution, the kernel would crash: > [ 122.154142] general protection fault, probably for non-canonical address 0x4141414141414141: 0000 [#1] PREEMPT SMP NOPTI > [ 122.156088] CPU: 0 PID: 917 Comm: kworker/0:6 Kdump: loaded Tainted: G W E 5.6.12-sevault+ #28 > [ 122.157855] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 > [ 122.159079] Workqueue: events_freezable_power_ disk_events_workfn > [ 122.160040] RIP: 0010:scsi_queue_rq+0x5af/0xa70 [scsi_mod] > [ 122.160916] Code: 01 89 83 9c 02 00 00 41 80 7f 08 00 74 07 83 8b 9c 02 00 00 08 48 8b 83 40 02 00 00 c7 83 3c 01 00 00 00 00 00 00 48 8d 78 08 <48> > c7 00 00 00 00 00 48 c7 40 58 00 00 00 00 48 83 e7 f8 48 29 f8 > [ 122.163821] RSP: 0018:ffffc900002efb08 EFLAGS: 00010202 > [ 122.164637] RAX: 4141414141414141 RBX: ffff888035b89c00 RCX: ffff888035b89ed0 > [ 122.165775] RDX: 0000000000000008 RSI: 0000000000000000 RDI: 4141414141414149 > [ 122.166891] RBP: ffff888035946000 R08: ffff888035a79860 R09: 0000000000000000 > [ 122.168016] R10: ffffea0001287280 R11: 0000000000000008 R12: ffff888035b89d18 > [ 122.169159] R13: ffff888035945000 R14: ffff888035946000 R15: ffffc900002efba0 > [ 122.170287] FS: 0000000000000000(0000) GS:ffff88807f800000(0000) knlGS:0000000000000000 > [ 122.171564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 122.172470] CR2: 0000560e654b77b8 CR3: 000000004dd38000 CR4: 00000000003406f0 > I believe the above example is without a SEV guest enabled/active, as SEV guest debugging can only be done with SEV Debug patches applied. > What and where gets overwritten entirely depends on what virtio driver is being > targeted. All manage their memory for the descriptor buffers differently so the overwrite > may require to be large. > > In the context of VirtIO and SWIOTLB, there are also these three fields other than the length: > dma_addr, flags, next > > I had a look around a little bit, so my take is the following: > > 1) There's already validation for dma_addr before doing the unmap with a call > to is_swiotlb_buffer (1). I think this check is sufficient. > > 2) flags > Before doing the unmap, the virtio implementation would check the flag and based on it > would select a DMA direction (TO/FROM DEVICE). Still, it seems that this would not > trick the driver to copy data to the device since only a `sync for CPU` may be performed > in the unmap path. That seems to be true. Thanks, Ashish [-- Attachment #1.2: Type: text/html, Size: 7291 bytes --] [-- Attachment #2: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-14 21:49 ` Konrad Rzeszutek Wilk (?) (?) @ 2020-12-15 8:47 ` Ashish Kalra 2020-12-15 10:54 ` Felicitas Hetzelt -1 siblings, 1 reply; 27+ messages in thread From: Ashish Kalra @ 2020-12-15 8:47 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: Thomas.Lendacky, Felicitas Hetzelt, Radev, Martin, david.kaplan, Michael S. Tsirkin, Jason Wang, Jon.Grimm, virtualization, Robert Buhren, iommu, brijesh.singh, Morbitzer, Mathias, hch On Mon, Dec 14, 2020 at 04:49:50PM -0500, Konrad Rzeszutek Wilk wrote: > On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > > Hello, > > Hi! Please see below my responses. > > > we have been analyzing the Hypervisor-OS interface of Linux > > and discovered bugs in the swiotlb/virtio implementation that can be > > triggered from a malicious Hypervisor / virtual device. > > With SEV, the SWIOTLB implementation is forcefully enabled and would > > always be used. Thus, all virtio devices and others would use it under > > the hood. > > > > The reason for analyzing this interface is that, technologies such as > > Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > > change the threat model assumed by various Linux kernel subsystems. > > These technologies take the presence of a fully malicious hypervisor > > into account and aim to provide protection for virtual machines in such > > an environment. Therefore, all input received from the hypervisor or an > > external device should be carefully validated. Note that these issues > > are of little (or no) relevance in a "normal" virtualization setup, > > nevertheless we believe that it is required to fix them if TDX or SNP is > > used. > > > > We are happy to provide more information if needed! > > > > [1] > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fcontent%2Fwww%2Fus%2Fen%2Fdevelop%2Farticles%2Fintel-trust-domain-extensions.html&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=THAJlYGLSOx3bKQYH62TLKH50By7Wnsu0z92snfNY84%3D&reserved=0 > > > > [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fen%2Fprocessors%2Famd-secure-encrypted-virtualization&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M3jmYCWaEvmAzIy%2F4z5XstsPf812SbEkuNX5PVVr0HY%3D&reserved=0 > > > > Bug: > > OOB memory write. > > dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > > and length parameters that are under control of the device. > > This happens e.g. in virtio_ring: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.10-rc7%2Fsource%2Fdrivers%2Fvirtio%2Fvirtio_ring.c%23L378&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j0CIi%2F8hBkVx45XGBtT4Ri52uWIOdOts%2BSbJ0kCB5B0%3D&reserved=0 > > Heya! > > Thank you for pointing this out! I've a couple of questions and hope you can > help me out with them. > > Also CC-ing AMD / TDX folks. > > Adding more relevant folks in AMD. Needless to say, the swiotlb code needs to validate this external untrusted input dma_addr and length parameters. Thanks, Ashish > > This raises two issues: > > 1) swiotlb_tlb_unmap_single fails to check whether the index generated > > from the dma_addr is in range of the io_tlb_orig_addr array. > > That is fairly simple to implement I would think. That is it can check > that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > is used. > > > 2) when swiotlb_bounce is called the device controls the length of the > > memory copied to the cpu address. > > So.. this sounds very similar to the Intel Thunder.. something issue > where this exact issue was fixed by handing the DMA off to the SWIOTLB > bounce code. > > But if that is broken, then that CVE is still not fixed? > > So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is > under the attacker control. Ugh. > > One way could be to have a io_tlb_orig_addr-ish array with the length > of mappings to double check? > > Couple more questions: > - Did you have already some PoC fixes for this? > - Is there a CVE associated with this? > - Is there a paper on this you all are working on? > > Thank you! _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-15 8:47 ` Ashish Kalra @ 2020-12-15 10:54 ` Felicitas Hetzelt 2020-12-15 14:37 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 27+ messages in thread From: Felicitas Hetzelt @ 2020-12-15 10:54 UTC (permalink / raw) To: Ashish Kalra, Konrad Rzeszutek Wilk Cc: Thomas.Lendacky, brijesh.singh, Radev, Martin, david.kaplan, Michael S. Tsirkin, Jason Wang, Jon.Grimm, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch Hello, thank you all for looking into this! To answer some of the questions: - Did you have already some PoC fixes for this: We don't have a full PoC or fix currently. Thought we have a PoC with which were able to overwrite memory outside of the mapped dma region. - Is there a CVE associated with this? No - Is there a paper on this you all are working on? Yes, we were planning to use this bug (among others in a paper) Could you point us to the intel thunder issue that you mentioned? On 12/15/20 9:47 AM, Ashish Kalra wrote: > On Mon, Dec 14, 2020 at 04:49:50PM -0500, Konrad Rzeszutek Wilk wrote: >> On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: >>> Hello, >> >> Hi! Please see below my responses. >> >>> we have been analyzing the Hypervisor-OS interface of Linux >>> and discovered bugs in the swiotlb/virtio implementation that can be >>> triggered from a malicious Hypervisor / virtual device. >>> With SEV, the SWIOTLB implementation is forcefully enabled and would >>> always be used. Thus, all virtio devices and others would use it under >>> the hood. >>> >>> The reason for analyzing this interface is that, technologies such as >>> Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] >>> change the threat model assumed by various Linux kernel subsystems. >>> These technologies take the presence of a fully malicious hypervisor >>> into account and aim to provide protection for virtual machines in such >>> an environment. Therefore, all input received from the hypervisor or an >>> external device should be carefully validated. Note that these issues >>> are of little (or no) relevance in a "normal" virtualization setup, >>> nevertheless we believe that it is required to fix them if TDX or SNP is >>> used. >>> >>> We are happy to provide more information if needed! >>> >>> [1] >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fcontent%2Fwww%2Fus%2Fen%2Fdevelop%2Farticles%2Fintel-trust-domain-extensions.html&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=THAJlYGLSOx3bKQYH62TLKH50By7Wnsu0z92snfNY84%3D&reserved=0 >>> >>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fen%2Fprocessors%2Famd-secure-encrypted-virtualization&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M3jmYCWaEvmAzIy%2F4z5XstsPf812SbEkuNX5PVVr0HY%3D&reserved=0 >>> >>> Bug: >>> OOB memory write. >>> dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr >>> and length parameters that are under control of the device. >>> This happens e.g. in virtio_ring: >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.10-rc7%2Fsource%2Fdrivers%2Fvirtio%2Fvirtio_ring.c%23L378&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j0CIi%2F8hBkVx45XGBtT4Ri52uWIOdOts%2BSbJ0kCB5B0%3D&reserved=0 >> >> Heya! >> >> Thank you for pointing this out! I've a couple of questions and hope you can >> help me out with them. >> >> Also CC-ing AMD / TDX folks. >>> > > Adding more relevant folks in AMD. > > Needless to say, the swiotlb code needs to validate this external untrusted input dma_addr and length parameters. > > Thanks, > Ashish > >>> This raises two issues: >>> 1) swiotlb_tlb_unmap_single fails to check whether the index generated >>> from the dma_addr is in range of the io_tlb_orig_addr array. >> >> That is fairly simple to implement I would think. That is it can check >> that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force >> is used. >> >>> 2) when swiotlb_bounce is called the device controls the length of the >>> memory copied to the cpu address. >> >> So.. this sounds very similar to the Intel Thunder.. something issue >> where this exact issue was fixed by handing the DMA off to the SWIOTLB >> bounce code. >> >> But if that is broken, then that CVE is still not fixed? >> >> So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is >> under the attacker control. Ugh. >> >> One way could be to have a io_tlb_orig_addr-ish array with the length >> of mappings to double check? >> >> Couple more questions: >> - Did you have already some PoC fixes for this? >> - Is there a CVE associated with this? >> - Is there a paper on this you all are working on? >> >> Thank you! _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length 2020-12-15 10:54 ` Felicitas Hetzelt @ 2020-12-15 14:37 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-15 14:37 UTC (permalink / raw) To: Felicitas Hetzelt, baolu.lu Cc: Thomas.Lendacky, Ashish Kalra, brijesh.singh, Radev, Martin, david.kaplan, Michael S. Tsirkin, Jason Wang, Jon.Grimm, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch On Tue, Dec 15, 2020 at 11:54:08AM +0100, Felicitas Hetzelt wrote: > Hello, > thank you all for looking into this! To answer some of the questions: > - Did you have already some PoC fixes for this: > We don't have a full PoC or fix currently. Thought we have a PoC > with which were able to overwrite memory outside of the mapped > dma region. > - Is there a CVE associated with this? > No > - Is there a paper on this you all are working on? > Yes, we were planning to use this bug (among others > in a paper) > > Could you point us to the intel thunder issue that you mentioned? ThunderClap was it! https://lwn.net/Articles/786558/ Cc-ing Lu Baolu .. Hm, this was a year ago and it looks like there are some extra SWIOTLB patches to be done ? > > On 12/15/20 9:47 AM, Ashish Kalra wrote: > > On Mon, Dec 14, 2020 at 04:49:50PM -0500, Konrad Rzeszutek Wilk wrote: > >> On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > >>> Hello, > >> > >> Hi! Please see below my responses. > >> > >>> we have been analyzing the Hypervisor-OS interface of Linux > >>> and discovered bugs in the swiotlb/virtio implementation that can be > >>> triggered from a malicious Hypervisor / virtual device. > >>> With SEV, the SWIOTLB implementation is forcefully enabled and would > >>> always be used. Thus, all virtio devices and others would use it under > >>> the hood. > >>> > >>> The reason for analyzing this interface is that, technologies such as > >>> Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > >>> change the threat model assumed by various Linux kernel subsystems. > >>> These technologies take the presence of a fully malicious hypervisor > >>> into account and aim to provide protection for virtual machines in such > >>> an environment. Therefore, all input received from the hypervisor or an > >>> external device should be carefully validated. Note that these issues > >>> are of little (or no) relevance in a "normal" virtualization setup, > >>> nevertheless we believe that it is required to fix them if TDX or SNP is > >>> used. > >>> > >>> We are happy to provide more information if needed! > >>> > >>> [1] > >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fcontent%2Fwww%2Fus%2Fen%2Fdevelop%2Farticles%2Fintel-trust-domain-extensions.html&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=THAJlYGLSOx3bKQYH62TLKH50By7Wnsu0z92snfNY84%3D&reserved=0 > >>> > >>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fen%2Fprocessors%2Famd-secure-encrypted-virtualization&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M3jmYCWaEvmAzIy%2F4z5XstsPf812SbEkuNX5PVVr0HY%3D&reserved=0 > >>> > >>> Bug: > >>> OOB memory write. > >>> dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > >>> and length parameters that are under control of the device. > >>> This happens e.g. in virtio_ring: > >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.10-rc7%2Fsource%2Fdrivers%2Fvirtio%2Fvirtio_ring.c%23L378&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j0CIi%2F8hBkVx45XGBtT4Ri52uWIOdOts%2BSbJ0kCB5B0%3D&reserved=0 > >> > >> Heya! > >> > >> Thank you for pointing this out! I've a couple of questions and hope you can > >> help me out with them. > >> > >> Also CC-ing AMD / TDX folks. > >>> > > > > Adding more relevant folks in AMD. > > > > Needless to say, the swiotlb code needs to validate this external untrusted input dma_addr and length parameters. > > > > Thanks, > > Ashish > > > >>> This raises two issues: > >>> 1) swiotlb_tlb_unmap_single fails to check whether the index generated > >>> from the dma_addr is in range of the io_tlb_orig_addr array. > >> > >> That is fairly simple to implement I would think. That is it can check > >> that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > >> is used. > >> > >>> 2) when swiotlb_bounce is called the device controls the length of the > >>> memory copied to the cpu address. > >> > >> So.. this sounds very similar to the Intel Thunder.. something issue > >> where this exact issue was fixed by handing the DMA off to the SWIOTLB > >> bounce code. > >> > >> But if that is broken, then that CVE is still not fixed? > >> > >> So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is > >> under the attacker control. Ugh. > >> > >> One way could be to have a io_tlb_orig_addr-ish array with the length > >> of mappings to double check? > >> > >> Couple more questions: > >> - Did you have already some PoC fixes for this? > >> - Is there a CVE associated with this? > >> - Is there a paper on this you all are working on? > >> > >> Thank you! _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: swiotlb/virtio: unchecked device dma address and length @ 2020-12-15 14:37 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 27+ messages in thread From: Konrad Rzeszutek Wilk @ 2020-12-15 14:37 UTC (permalink / raw) To: Felicitas Hetzelt, baolu.lu Cc: Thomas.Lendacky, Ashish Kalra, brijesh.singh, Radev, Martin, david.kaplan, Michael S. Tsirkin, Jon.Grimm, virtualization, Robert Buhren, iommu, Morbitzer, Mathias, hch On Tue, Dec 15, 2020 at 11:54:08AM +0100, Felicitas Hetzelt wrote: > Hello, > thank you all for looking into this! To answer some of the questions: > - Did you have already some PoC fixes for this: > We don't have a full PoC or fix currently. Thought we have a PoC > with which were able to overwrite memory outside of the mapped > dma region. > - Is there a CVE associated with this? > No > - Is there a paper on this you all are working on? > Yes, we were planning to use this bug (among others > in a paper) > > Could you point us to the intel thunder issue that you mentioned? ThunderClap was it! https://lwn.net/Articles/786558/ Cc-ing Lu Baolu .. Hm, this was a year ago and it looks like there are some extra SWIOTLB patches to be done ? > > On 12/15/20 9:47 AM, Ashish Kalra wrote: > > On Mon, Dec 14, 2020 at 04:49:50PM -0500, Konrad Rzeszutek Wilk wrote: > >> On Fri, Dec 11, 2020 at 06:31:21PM +0100, Felicitas Hetzelt wrote: > >>> Hello, > >> > >> Hi! Please see below my responses. > >> > >>> we have been analyzing the Hypervisor-OS interface of Linux > >>> and discovered bugs in the swiotlb/virtio implementation that can be > >>> triggered from a malicious Hypervisor / virtual device. > >>> With SEV, the SWIOTLB implementation is forcefully enabled and would > >>> always be used. Thus, all virtio devices and others would use it under > >>> the hood. > >>> > >>> The reason for analyzing this interface is that, technologies such as > >>> Intel's Trusted Domain Extensions [1] and AMD's Secure Nested Paging [2] > >>> change the threat model assumed by various Linux kernel subsystems. > >>> These technologies take the presence of a fully malicious hypervisor > >>> into account and aim to provide protection for virtual machines in such > >>> an environment. Therefore, all input received from the hypervisor or an > >>> external device should be carefully validated. Note that these issues > >>> are of little (or no) relevance in a "normal" virtualization setup, > >>> nevertheless we believe that it is required to fix them if TDX or SNP is > >>> used. > >>> > >>> We are happy to provide more information if needed! > >>> > >>> [1] > >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fcontent%2Fwww%2Fus%2Fen%2Fdevelop%2Farticles%2Fintel-trust-domain-extensions.html&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=THAJlYGLSOx3bKQYH62TLKH50By7Wnsu0z92snfNY84%3D&reserved=0 > >>> > >>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fen%2Fprocessors%2Famd-secure-encrypted-virtualization&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M3jmYCWaEvmAzIy%2F4z5XstsPf812SbEkuNX5PVVr0HY%3D&reserved=0 > >>> > >>> Bug: > >>> OOB memory write. > >>> dma_unmap_single -> swiotlb_tbl_unmap_single is invoked with dma_addr > >>> and length parameters that are under control of the device. > >>> This happens e.g. in virtio_ring: > >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.10-rc7%2Fsource%2Fdrivers%2Fvirtio%2Fvirtio_ring.c%23L378&data=04%7C01%7Cashish.kalra%40amd.com%7C1d1cbca182a84c0e504708d8a079eec0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637435792867090126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j0CIi%2F8hBkVx45XGBtT4Ri52uWIOdOts%2BSbJ0kCB5B0%3D&reserved=0 > >> > >> Heya! > >> > >> Thank you for pointing this out! I've a couple of questions and hope you can > >> help me out with them. > >> > >> Also CC-ing AMD / TDX folks. > >>> > > > > Adding more relevant folks in AMD. > > > > Needless to say, the swiotlb code needs to validate this external untrusted input dma_addr and length parameters. > > > > Thanks, > > Ashish > > > >>> This raises two issues: > >>> 1) swiotlb_tlb_unmap_single fails to check whether the index generated > >>> from the dma_addr is in range of the io_tlb_orig_addr array. > >> > >> That is fairly simple to implement I would think. That is it can check > >> that the dma_addr is from the PA in the io_tlb pool when SWIOTLB=force > >> is used. > >> > >>> 2) when swiotlb_bounce is called the device controls the length of the > >>> memory copied to the cpu address. > >> > >> So.. this sounds very similar to the Intel Thunder.. something issue > >> where this exact issue was fixed by handing the DMA off to the SWIOTLB > >> bounce code. > >> > >> But if that is broken, then that CVE is still not fixed? > >> > >> So the issue here is that swiotlb_tbl_unmap_single(..,mapping_size,) is > >> under the attacker control. Ugh. > >> > >> One way could be to have a io_tlb_orig_addr-ish array with the length > >> of mappings to double check? > >> > >> Couple more questions: > >> - Did you have already some PoC fixes for this? > >> - Is there a CVE associated with this? > >> - Is there a paper on this you all are working on? > >> > >> Thank you! _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2020-12-18 9:29 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-11 17:31 swiotlb/virtio: unchecked device dma address and length Felicitas Hetzelt 2020-12-14 21:49 ` Konrad Rzeszutek Wilk 2020-12-14 21:49 ` Konrad Rzeszutek Wilk 2020-12-15 3:20 ` Jason Wang 2020-12-15 3:20 ` Jason Wang 2020-12-15 14:27 ` Konrad Rzeszutek Wilk 2020-12-15 14:27 ` Konrad Rzeszutek Wilk 2020-12-16 5:53 ` Jason Wang 2020-12-16 5:53 ` Jason Wang 2020-12-16 6:41 ` Jason Wang 2020-12-16 6:41 ` Jason Wang 2020-12-16 13:04 ` Konrad Rzeszutek Wilk 2020-12-16 13:04 ` Konrad Rzeszutek Wilk 2020-12-17 4:19 ` Jason Wang 2020-12-17 4:19 ` Jason Wang 2020-12-17 22:55 ` Ashish Kalra 2020-12-16 8:54 ` Michael S. Tsirkin 2020-12-16 8:54 ` Michael S. Tsirkin 2020-12-16 13:07 ` Konrad Rzeszutek Wilk 2020-12-16 13:07 ` Konrad Rzeszutek Wilk 2020-12-16 22:07 ` Radev, Martin 2020-12-17 23:17 ` Ashish Kalra 2020-12-18 9:28 ` Radev, Martin 2020-12-15 8:47 ` Ashish Kalra 2020-12-15 10:54 ` Felicitas Hetzelt 2020-12-15 14:37 ` Konrad Rzeszutek Wilk 2020-12-15 14:37 ` Konrad Rzeszutek Wilk
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.