The site is required to allow the building of a new link road between Edgar Street and Commercial Road for which planning permission was granted in 2010.
The road is part of the wider regeneration plan for the city and is intended to remove traffic from the existing inner ring road and allow access to 800 new homes.
The agreement follows lengthy negotiations over several years.
Faced with the prospect of a Compulsory Purchase Order, a negotiated settlement has been reached with Rockfield, owned by the Wolverson family.
Leon Wolverson, managing director of Rockfield, said: “It is with great sadness that after lengthy negotiations we have been unable to find a suitable relocation site in the city and therefore we will not be relocating the business.
“Priority will now be given to consultation with staff, but the store as part of the terms agreed with the Council will continue to trade for the immediate future.”
Cllr Graham Powell, the council’s cabinet member for highways, said: “This is a significant step in the regeneration of Hereford City centre.
“This acquisition plays a key role in improving transport in Hereford and in taking traffic away from the city centre.”