Coronavirus disease 2019 (COVID-19) spawned a global health crisis in late 2019 and is caused by the novel coronavirus SARS-CoV-2. SARS-CoV-2 infection can lead to elevated markers of endothelial dysfunction associated with higher risk of mortality. It is unclear whether endothelial dysfunction is caused by direct infection of endothelial cells or is mainly secondary to inflammation. Here, we investigate whether different types of endothelial cells are susceptible to SARS-CoV-2. Human endothelial cells from different vascular beds including umbilical vein endothelial cells, coronary artery endothelial cells (HCAEC), cardiac and lung microvascular endothelial cells, or pulmonary arterial cells were inoculated in vitro with SARS-CoV-2. Viral spike protein was only detected in HCAECs after SARS-CoV-2 infection but not in the other endothelial cells tested. Consistently, only HCAEC expressed the SARS-CoV-2 receptor angiotensin-converting enzyme 2 (ACE2), required for virus infection. Infection with the SARS-CoV-2 variants B.1.1.7, B.1.351, and P.2 resulted in significantly higher levels of viral spike protein. Despite this, no intracellular double-stranded viral RNA was detected and the supernatant did not contain infectious virus. Analysis of the cellular distribution of the spike protein revealed that it co-localized with endosomal calnexin. SARS-CoV-2 infection did induce the ER stress gene EDEM1, which is responsible for clearance of misfolded proteins from the ER. Whereas the wild type of SARS-CoV-2 did not induce cytotoxic or pro-inflammatory effects, the variant B.1.1.7 reduced the HCAEC cell number. Of the different tested endothelial cells, HCAECs showed highest viral uptake but did not promote virus replication. Effects on cell number were only observed after infection with the variant B.1.1.7, suggesting that endothelial protection may be particularly important in patients infected with this variant.