The Tynecastle club face Rangers in the Betfred Cup semi-final at Hampden on Sunday but have won just once in the Ladbrokes Premiership this season.
Levein had been in charge since replacing Ian Cathro in August 2017 but now moves on, with his assistant Austin MacPhee placed in interim charge.
A club statement read: "The Board would like to thank Craig for all of his efforts as first-team manager. Craig has worked tirelessly and selflessly, around the clock, to try to achieve success, despite the many challenges with which he has been faced."
