Riina, who had been serving 26 life sentences and is thought to have ordered more than 150 murders, had been in a medically-induced coma after his health deteriorated following two operations.
The mobster, who turned 87 on Thursday, died in the prisoners' wing of a hospital in Parma in northern Italy just before 4am, according to the country's main dailies and Ansa news agency.
Nicknamed "The Beast" because of his cruelty, Salvatore "Toto" Riina led a reign of terror for decades after taking control of the island's powerful organised crime group Cosa Nostra in the 1970s.
The most high-profile murders he ordered were those in 1992 of anti-mafia judges Giovanni Falcone and Paolo Borsellino, who had worked fearlessly to bring more than 300 mobsters to trial in 1987.
The remains of a police car destroyed during of the assassination of judge Giovanni Falcone, now a memorial in Palermo. Photo: Marcello Paternostro/AFP
He also famously ordered the brutal murder of a 13-year-old boy who was kidnapped in a bid to stop his father from spilling mafia secrets. The boy was strangled and his body dissolved in acid.
"God have mercy on him, as we won't," an association for victims told the Fatto Quotidiano daily.
Riina's family had been given permission by Italy's health ministry on Thursday for a rare visit to say goodbye.
"You're not Toto Riina to me, you're just my dad. And I wish you happy birthday dad on this sad but important day, I love you," one of his sons, Salvatore, wrote on Facebook.
The ageing gangster had asked in July to be released from prison on the grounds of serious illness – a request that was denied after a court ruled the care he received behind bars was as good as he would get on the outside.
Doctors said at the time that the former boss was "lucid". He was caught on a wiretap earlier this year saying he "regrets nothing... They'll never break me, even if they give me 3,000 years" in jail.
Toto Riina (left) and Bernardo Provenzano, who replaced Riina at the head of the mafia. Photo: handout/Grazia Neri/AFP
The son of a poor farmer, he was born on November 16th 1930 in Corleone, a village near Palermo that would become synonymous with the mafia thanks to Francis Ford Coppola's Godfather film trilogy.
He lost his father and a brother young when they were blown up trying to extract gunpowder from an unexploded American bomb in 1943.
By the time he was 19, he had killed his first victim.
He started off as a foot soldier to boss Luciano Leggio before moving up through the ranks, going on the run in 1969 but continuing to lead first the Corleone clan then the entire mafia from hiding.
He would elude police efforts to snare him for almost a quarter of a century, without ever leaving Sicily.
Riina was slapped with multiple life sentences in absentia after a fellow mobster turned state witness.
The "Beast" retaliated by ordered the deaths of 11 of the turncoat's relatives. He was eventually captured in 1993 after a tip-off from a rival.
Rina's home in Corleone, Sicily, was confiscated and turned into a B&B. Photo: Marcello Paternostro/AFP