A high pressure center is where the pressure has been measured to be the highest relative to its surroundings. That means, moving in any direction away from the "High" will result in a decrease in pressure. A high pressure center also represents the center of an anticyclone and is indicated on a weather map by a blue "H".
Winds flow clockwise around a high pressure center in the northern hemisphere, while in the southern hemisphere, winds flow counterclockwise around a high.
Sinking air in the vicinity of a high pressure center suppresses the upward motions needed to support the development of clouds and precipitation. This is why fair weather is commonly associated with an area of high pressure.