Hampton Court in Herefordshire had many attributes that appealed to him. The imposing 19th century castle, with far earlier origins, was designed by a gentleman architect for the Arkwright family of Spinning Jenny fame. Its name may have added extra romance because of its reminder of London's Hampton Court Palace, but as far as we know, one is not the other's namesake.

Early in 1994, Van Kampen was keen enough, based on what he had heard about the property, to bid for it sight unseen from where he lived in Michigan but his offer was not accepted. However in May of that year, he and his wife Judith, visited England and, popping in to see what they had missed, fell seriously in love with Hampton's tattered charms.

Scott Pierre, married to the Van Kampen's daughter, Karla, recalls what happened when his parents-in-law discovered that Hampton Court had still not been sold. "Bob sat down with Mrs Folkes [the wife of the then owner of Hampton Court]. She rang her husband, who was frustrated at the situation. He came home. My father-in-law saw his opportunity and, in 10 minutes, they had a deal. The house, together with 1,000 acres - later sold on to local farmers - went for around Knight Frank's guide price of £5 million.

So, almost by accident, the Van Kampens had finally bought a huge Herefordshire farming estate. 20 acres of parkland plus a house in very poor shape and a garden that required nothing but constant grass cutting by three field mowers, because it was nothing but grass. That was in May 1994. In September of that year the whole family, Robert, Judith, Scott and Karla arrived in Herefordshire to decide what to do with their new white elephant. It was the start of one of the most spectacular acts of private philanthropy in recent years in this country, resulting in the almost magical transformation of the estate.

Robert Van Kampen was the founder of Van Kampen Merrit, an investment banking firm that he sold to Xerox in 1984 for about $200 million. The family now controls assets of about $79 billion and is lavishing £12 million on the restoration of Hampton Court and its gardens and parkland. But they have no plans to live at the house and regard the whole project as the fulfilment of Robert Van Kampen's lifelong dream and interests.

Without any plans in the initial stages to develop the gardens, they began with a huge programme of repair to the Grade 1 listed house itself. Its battlements and mullions of Forest-of-Dean sandstone were crumbling, its roof was a goner. As the smart new guidebook points out laconically: "It was in a state of some disrepair, the last major work having been carried out… 160 years before".

"We put 80 tons of new lead on the roof and 55,000 slates. The whole was completely scaffolded and two of the towers were replaced. Our mandate has been to protect it for 25 to 30 years Pierre explains. A caretaker, Edward Waghorn, was appointed and later became manager. In five years, he has overseen its new roof, the gradual replacement of the stonework and complete rewiring of the house. Then came the garden, almost as an afterthought. "A year after we started with the restoration, I had an idea that we could do much more," says Waghorn. "The Van Kampens are an extraordinary family - they give people opportunities and are very open to possibilities." So he asked if he could embark on a project to recreate the gardens. When the family agreed, he found a designer in Simon Dorrell, the art editor of the gardening magazine Hortus and - with its owner and editor, David Wheeler - the designer of their famous garden at Bryansground nearby.

Dorrell came up with a grand-scale plan, in keeping with the house. It had avenues, canals, a maze, a tunnel and a sunken garden, together with pavilions and belvederes. Waghorn took the entire scheme to the Van Kampens in the spring of 1996. They simply said "yes" to the whole project. "They just fell in love with it," says Waghorn, "It took us all rather by surprise. We've done it stage by stage, but it has been a major investment". It was at this time that the decision was taken to open the gardens to the public.

Waghorn says that he keeps in touch almost daily with the family who control all the expenditure tightly. "They allowed the garden to happen but they're very careful about major expenditure and how savings can be made". Everyone is coy about the actual cost. But the restoration of the house is reckoned to have already eaten up about £6 million and the gardens will probably consume not much less, especially as a stone-built gatehouse, a hermitage and a stone tower have been built from scratch.

Waghorn, however, says that costs were kept down by hiring full-time estate staff, rather than contractors. The policy is to employ as many local people as possible. This is a source of special satisfaction to Scott Pierre who tells me: "Everything that has been done is first class, and we haven't had to go to London or New York."

Jim Howden, the head mason, has been working for four years on the battlements and roof: "The Van Kampens don't have a carpet-bagging approach. My team is totally committed. It's unusual to have a team dedicated to one building but it was done with Worcester and Gloucester cathedrals." Exactly so.

It was Howden's team which constructed the new tower that Dorrell designed to be the centre of the maze. He put the tower there as a cunning way to get people into the adjoining sunken garden. They are led to the top of the tower with its vista of much of the garden and down circular stone steps into a basement. The visitor then stumbles along a dark tunnel into a grotto and thatched hermitage before coming out into the shady green sunken area.

The hermitage, and all the many wooden structures appearing in the gardens, were made by the head carpenter, Geoff Williams, and his team. He arrived at Hampton Court three and a half years ago, first repairing a medieval cross-passage screen in the house, a vestige of an earlier building. The team recently set about creating the two octagonal oak pavilions designed by Dorrell at the heart of the newly completed walled garden, and is now restoring the first of three Victorian greenhouses. "When we've finished that, Ed's keen for us to make a bridge over the River Lugg and restore the pumphouse."

Williams's group may be small but it's impressive: four masons, four carpenters, four gardeners, a bricklayer and two all-round building workers, and, of course, Simon Dorrell with planting advice from David Wheeler. Then there was Johnny Felton who spent six months creating and restoring the drystone wall haha around the main garden. Everything has been made as close as possible to the original - a blacksmith is employed virtually full time to make iron fittings.

Of the 100 acres which came with the house, about 12 are parkland outside the haha and eight make up the garden proper - the extensive remaining acreage is agricultura1 land. The primary source of Dorrell's inspiration is a garden designed for Hampton by George London in the l69Os, although Humphry Repton's lake of later date in the park has been dredged. "London's garden had four miles of avenues. We intend to plant about two miles."

Dorrell's plans hark back - but in a 21st-century manner. The walled garden within high brick walls created by the Arkwrights has its share of canals, but these bubble sensuously, or slide down stone flagged slopes. The whole is reached through a tunnel of aged wisterias which drip with flowers in early summer and open on to a deliberately surprising double herbaceous border, planted with the lushness - 'full blown and romantic', Dorrell calls it - so associated with Bryansground. The scheme here is of purple and yellow, realised in veronicas and delphiniums, oregano and achillea.

The plan for the main area of walled garden is extremely complex. Two formal pavilions, which will be festooned eventually with the rampant white roses, "Rambling Rector" and "The Garland", rest in still pools of water with a different view from every side. The colour scheme is the same throughout - blue, maroon, grey, silver, white and pink - but achieved differently, maybe with massed lavender ('Little Lady'), or formal silver pear trees. "It is eight gardens in one," says Dorrell, "I've designed it as a series of small gardens an interlocking jigsaw."

A new Dutch garden is being built nearby. From a tennis court, with the 1920s pavilion left intact at the head of a still canal, rows of box and Irish yews have been planted in a pattern inspired by the ballroom ceiling in the house. Agapanthus is planted in large terracotta pots made by a local craftsman, Mark Griffiths.

The Sunken Garden is also near completion. "This is an area Arkwright would recognise, though when I saw it, it was filled with saplings and was a rubbish tip," recollects Dorrell. A pond is fed by a small waterfall which you can walk behind. The beds, which rise in terraces from the water, have been planted with shade lovers and, unlike the rest of the garden, this has an air of Victorian melancholy.

Dorrell believes there are enough ongoing projects to last for years. As well as restoring more avenues at the front of the house, which is currently just lawn and specimen trees, the Joseph Paxton conservatory will be rescued and a vegetable garden planted in a second walled garden.

Private munificence on such a sale invites the question - why? A partial answer can be found in Sola Scriptura, the non-profit making Christian trust founded by Van Kampen to affirm the authority of the Bible, which owns Hampton Court.

Van Kampen was a Christian fundamentalist, with a famous collection of biblical manuscripts. He was also known for applying biblical strictures to the running of his business and there was a strict code of personal conduct among his many employees: divorce was frowned on and hard liquor-drinking discouraged. Hampton Court, which is always referred to by Scott Pierre as "the facility", is being developed as a temperance conference centre, in line with the founder's beliefs. When I visited this summer, the house was full of young Americans - friends of the Sola Scriptura - on retreat.

Scott Pierre, who runs the trust, remarks, "We are a family with strong Christian roots and after all, Christianity began in a garden." Unfortunately, Robert Van Kampen died, aged 60, in October last year, awaiting a heart transplant, a few months before the Hampton Court project was launched to the public. "I don't think any of us, including Bob, had any idea of the scope of the place and what work we had undertaken," Pierre says. "If we had, we might never have done it. But now we have, we are delighted and proud of what the team has achieved. My only regret is that Bob couldn't be here to see it."