The social sciences today are built around collecting data. Polls seem very important. Opinions are gathered and regressions are performed and truth is finally synthesized. Except, there is a problem with social experiments: namely, they have proven quite difficult to replicate.
Much of what is being done in the social sciences today is what Steve Sailer aptly points out is marketing research. The reason that social experiments can't be repeated is because they have only spotted a particular trend, not identified a truth about mankind. This is a problem on several fronts, for instance: 1) predictable human behavior is often used to justify state force in the marketplace and 2) the people most interested in justifying state force are in position to fund or subsidize research.
Spotting trends is important for marketing campaigns aimed at reminding people to buy products or to convince new people to buy products. As a justification for state force: well... just imagine cigarette commercials from the early 20th century as being metaphors for political policies today.
Should policies be based on trends? In the early 2000s, housing prices maintained the same trajectory that they had been trending for decades. Understanding that this trend reflected a general truth about housing, policies were put in place to open access to mortgages for people who otherwise could not achieve the American Dream. Under the ostensible impetus of increasing drug crime, policies emerged to combat this rising trend. The housing policies turned out to be an incredible boon for people with variable rate mortgages and the economy in general; and, the drug crime of the late 80s has virtually been eradicated. And smoking Lucky Brand cigarettes builds vigor.
Predicting the future, on the other hand, tends to be a bit of a ball-and-cup game. Some people continue to make the same predictions, only to be proven right like a broken clock: the doom-and-gloom forecasters, the benevolent bureaucrats, the commodity analysts. These people have a consistent message and when they are right, you're wrong; and when they're wrong, it was a timing issue.
Science fiction has a had a pretty good track record of predicting technologies, institutions and cultural turns. Jules Verne foresaw the submarine and the scuba tank; Arthur C. Clarke predicted the sling shot maneuver; Phillip Dick presaged the pre-crime of DUI checkpoints and the various other guilty until proven innocent show trials of the modern day.
On another spectrum entirely are economists: part seers, part scientists. Cantillon described the process by which devalued money drives overvalued money under the mattress. This allows us to make fairly accurate predictions given a particular circumstance that it describes. Same thing with supply and demand curves. Perhaps the most famous (least talked about) prediction is that of von Mises logical assessment of the decline of the Soviet Union. While other prominent economists praised the foresight of a command economy, Mises explained that the lack of market feedback in terms of pricing would inevitably crush the Soviet Union. But that took 70 years.
Can we really say that people have had any success predicting the future? And if they haven't, how should our institutions reflect this general truth? Mises was right about the Soviet Union specifically; but his general ideas are rarely given proper, much less short, shrift. Complex economies require freedom of action to establish prices; prices bake a pretty complex cake with ingredients about past behaviors, present behaviors and possible future ones. Hayek called this process decentralized knowledge. Under this viewpoint, the future is unknown and ideas compete to allocate resources to satisfy future wants. The ideas of Mises and Hayek predict a general rise in living standards concomitant with freedom of prices, which appear to correspond to societies operating under those policies. Am I hedging or is that about as close a description of a natural phenomena that we can get?