Saif al-Islam was killed in a "treacherous and cowardly" act, in which four masked men stormed his residence late on Tuesday in the western Libyan city of Zintan, his political office said.
His lawyer Khaled al-Zaidi confirmed to dpa that Saif al-Islam Gadhafi was "assassinated at his home".
The al-Arabiya news channel reported that the 53-year-old was shot dead in the garden of his residence in Zintan, citing sources close to the Gadhafi family.
Saif al-Islam was the second-eldest son of the long-time Libyan leader, who ruled the North African country for more than four decades.
Before his father's overthrow and death in 2011, al-Islam led a reform project aimed at modernising the country politically and bringing it closer to the West.
Many of these efforts were soon reversed in order not to upset the balance of power in his father's government.
Gadhafi's death followed months of mass protests in 2011. Saif al-Islam supported the brutal crackdown of the uprising.
While attempting to flee to Niger, he was captured by a militia and imprisoned in Zintan. According to his own account, he spent several years there with little contact with the outside world.
The International Criminal Court has been seeking Saif al-Islam's extradition since 2014 in order to try him for crimes against humanity during the uprisings in Libya.
In 2015, he was sentenced to death in absentia by a court in Libya's capital, Tripoli. The rulers in Zintan neither transferred him to Tripoli nor extradited him.
In 2021, he reappeared unexpectedly after a long absence and submitted his candidacy for the planned presidential election in Libya.
The election never took place due to a conflict over the constitutional basis and the eligible candidates. The country is now effectively divided between two hostile governments in the east and west.