What exactly is the difference between .NET Core and ASP.NET Core?

.NET Core is a runtime. It can execute applications that are built for it.

ASP.NET Core is a collection of libraries that form a Framework for building web applications. ASP.NET Core libraries can be used on both .NET Core and the “Full .NET Framework” (what has shipped with windows for many years).