The first fortifications in Japan were made primarily of earthworks, or rammed earth, and wood, but they made far greater use of natural defences and topography than anything man-made. These castles were never intended to be long-term defensive positions: they were built when needed and abandoned afterwards.

The Yamato people began to build cities in earnest in the 7th century, complete with expansive palace complexes, surrounded on four sides with walls and impressive gates. Earthworks and wooden fortresses were also built throughout the countryside to defend the territory from the native Emishi, Ainu and other groups; unlike their primitive predecessors, these were relatively permanent structures, built in peacetime. These were largely built as extensions of natural features, and often consisted of little more than earthworks and wooden barricades.

The Heian period (794–1185) saw a shift from the need to defend the entire state from invaders to that of lords defending individual mansions or territories from one another. The rise of the samurai warrior class towards the end of the period, and various disputes between noble families jostling for power and influence in the Imperial Court, brought about further upgrades. As factions emerged and loyalties shifted, clans and factions that had helped the Imperial Court became enemies, and defensive networks were broken or altered through the shifting of alliances.

Fortifications were still made almost entirely out of wood, and largely based on earlier modes, and on Chinese and Korean examples. But they began to become larger, to incorporate more buildings, to accommodate larger armies, and to be conceived as more long-lasting structures. Chihaya Castle and Akasaka Castle, permanent castle complexes containing a number of buildings but no tall keep towers, and surrounded by wooden walls, were built by to be as militarily effective as possible, within the technology and designs of the time. Castle complexes became fairly elaborate, containing a number of structures, some of which were quite complex internally, as they now served as residences, command centres, and for other purposes.

The Ōnin War, which broke out in 1467, marked the beginning of nearly 150 years of widespread warfare (called the Sengoku period) between daimyōs (feudal lords) across the entire archipelago. Noble family mansions across the city became increasingly fortified, and attempts were made to isolate Kyoto from the marauding armies of samurai that dominated the landscape. Over the course of the Sengoku period, many mountain castles developed into permanent residences, with elaborate exteriors and lavish interiors.

The beginnings of the shapes and styles now considered to be the "classic" Japanese castle design emerged at this time, and castle towns (jōkamachi, "town below castle") also appeared and developed. In the last thirty years of this period, drastic changes occurred to bring about the type of castle typified by Himeji Castle and other surviving castles, along with the introduction of firearms and the development of tactics to employ or counter them.

Japanese castle-building was spurred by the introduction of firearms. Though firearms first appeared in Japan in 1543 and castle design almost immediately changed in response, Azuchi Castle, built in the 1570s, was the first example of a largely new type of castle, on a larger, grander scale than those that came before, boasting a large stone base (武者返し, musha-gaeshi), a complex arrangement of concentric baileys (丸, maru), and a tall central tower. In addition, the castle was located on a plain, rather than on a densely forested mountain, and relied more heavily on architecture and man-made defenses than on its natural environment for protection. These features, along with the general appearance and organization of these buildings, have come to define the stereotypical Japanese castle.

Azuchi Castle's stone foundation resisted damage from arquebus balls better than wood or earthworks, and the larger scale of the complex added to the difficulty of destroying it. Tall towers and the castle's location on a plain provided greater visibility from which the garrison could employ their guns, and the complex set of courtyards and baileys provided additional opportunities for defenders to retake portions of the castle that had fallen.

Cannons were rare in Japan because they had to be purchased at great expense from foreigners, as the foundries used to make bronze temple bells were unsuited to the production of iron or steel cannons. The few cannons that were used were smaller and weaker than those used in European sieges, and many of them were in fact taken from European ships and remounted to serve on land; whereas the advent of the cannon and other artillery brought an end to stone castles in Europe, wooden ones would remain in Japan for several centuries longer. It was often seen to be more honorable, and more tactically advantageous on the part of the defender, for him to lead his forces into battle outside the castle. When battles were not resolved in this way, out in the open, sieges were almost always undertaken purely by denying supplies to the castle, an effort that could last years but involved little more than surrounding the castle with a force of sufficient size until a surrender could be elicited.

The crucial development that spurred the emergence of a new type of defensive architecture was the advent of firearms. Arquebus firing squads and cavalry charges could overcome wooden stockades with relative ease, so stone castles came into use.

Among the many castles built in the ensuing years was Hideyoshi's castle at Osaka, completed in 1585. It incorporated all the new features and construction philosophies of Azuchi, and was larger, more prominently located, and longer-lasting. It was the last bastion of resistance against the establishment of the Tokugawa Shogunate (see Siege of Osaka) and remained prominent although no longer politically or militarily significant, as the city of Osaka grew up around it.

The Edo period ushered in over 250 years of peace, beginning around 1600–1615 and ending in 1868. Edo period castles no longer had defense against outside forces as their primary purpose. Rather, they served primarily as luxurious homes for the daimyōs, their families and retainers, and to protect the daimyō and his power base against peasant uprisings and other insurrections. The architectural style did not change much from more martial times, but the furnishings and indoor arrangements could be quite lavish.

This restriction on the number of castles allowed each han had profound effects — not only politically, as intended, but socially, and in terms of the castles themselves. Whereas members of the samurai class had previously lived in or around the great number of castles sprinkling the landscape, they now became concentrated in the capitals of the han and in Edo. Meanwhile, the castles in the han capitals expanded to accommodate the increased number of samurai they now had to support and also to represent the prestige and power of the daimyō, now consolidated into a single castle.

All castles, along with the feudal domains themselves, were turned over to the Meiji government in the 1871 abolition of the han system. During the Meiji Period, these castles were viewed as symbols of the previous ruling elite and impediments to modernization, so nearly 2,000 castles were dismantled or destroyed. Others were abandoned and eventually fell into disrepair.

Some castles, especially the larger ones, were used by the Imperial Japanese Army. Osaka Castle served as the headquarters for the 4th Infantry Division. Hiroshima Castle served as Imperial General Headquarters during the First Sino-Japanese War (1894–1895) and later as the headquarters for the 5th Infantry Division; Kanazawa Castle served as HQ for the 9th Infantry Division. For this reason, and as a way to strike against the morale and culture of the Japanese people, many castles were bombed during World War II. The main towers of the castles at Nagoya, Osaka, Okayama, Fukuyama, Wakayama, and Ōgaki, among others, were all destroyed during air raids. Hiroshima Castle is notable for having been destroyed in the atomic bomb blast on August 6, 1945.

During the early 20th century, new laws were introduced for the preservation of heritage. Local governments had a legal obligation to prevent any further destruction, and they had some funds from the national government to improve these historically significant sites.

By the 1920s, nationalism was on the rise, along with a new pride in the castles, which became symbols of Japan's warrior traditions. With new advances in construction, some of the previously destroyed castles were re-built quickly and cheaply with steel-reinforced concrete.

While many of the remaining castles in Japan are reconstructions, and most of these are steel-reinforced concrete replicas, there has been a movement toward traditional methods of construction. Kanazawa Castle is a remarkable example of a modern reproduction using a significant degree of traditional construction materials and techniques. Modern construction materials at Kanazawa Castle are minimal, discreet, and are primarily in place to ensure stability, safety and accessibility.

Most of the 12 original castles are in areas of Japan that were not subjected to strategic bombing in World War II, such as in Shikoku or the Japanese Alps.

At the other end of the spectrum are castles that have been left in ruins, though usually after archaeological surveys and excavations were done. Most of these belong to or are maintained by local municipal governments. Some have been incorporated into public parks. Others have been left in a more natural state, often with a marked hiking trail. The grounds of some were developed with municipal buildings or schools.

Some castle sites are now in the hands of private landowners, in areas that have been developed. Vegetable plots now occupy the site of Kaminogo Castle (Gamagōri, Aichi), and a chestnut orchard has been planted on the site of Nishikawa Castle, though in both cases some of the castle-related topography can still be seen, such as the motte or ramparts.

Finally there are the castle sites that have not been maintained or developed to any degree, and may have few markings or signs. Historical significance and local interest are too low to warrant additional costs. Castle sites of this type also include nearly every area marked "Castle Mountain" (城山 Shiroyama) on the maps of towns and cities across Japan. Because the castle was small or may have been used for a short time in centuries past, the name of the castle is often lost to history, such as the "Shiroyama" at Sekigahara, Gifu Prefecture, or the "Shiroyama" between Lake Shōji and Lake Motosu near Mount Fuji, Yamanashi Prefecture. In such cases, locals might not be aware there ever was a castle, believing that the name of the mountain is "just a name". Detailed city maps will often have such sites marked. At the site, castle-related landscaping, such as ramparts, partly filled wells, and a leveled hilltop or series of terraces, will provide evidence of the original layout of the castle.

Many castles across Japan serve as history and folk museums, as points of pride for local people, and as tangible structures reflecting Japanese history and heritage. As castles are associated with the martial valor of past warriors, there are often monuments near castle structures or in their parks dedicated to samurai or soldiers of the Imperial Army who died in war, such as the monument to the 18th Infantry Regiment near the ruins of Yoshida Castle (Toyohashi, Aichi). Castle grounds are often developed into parks for the benefit of the public, and planted with cherry blossom trees, plum blossom trees and other flowering plants. Hirosaki Castle in Aomori Prefecture and Matsumae Castle in Hokkaido are both famous in their respective regions for their cherry blossom trees. The government and dedicated groups have made efforts to keep castles relevant and visible in the lives of the Japanese people, to showcase them to visitors, and to prevent the neglect of national heritage.

Japanese castles were almost always built atop a hill or mound, and often an artificial mound would be created for this purpose. This not only aided greatly in the defense of the castle, but also allowed it a greater view over the surrounding land, and made the castle look more impressive and intimidating. The use of stone, and the development of the architectural style of the castle, was a natural step up from the wooden stockades of earlier centuries. The hills gave Japanese castles sloping walls, which many argue helped (incidentally) to defend them from Japan's frequent earthquakes.

Techniques were invented to keep attackers off the walls and to stop them from climbing the castle, including pots of hot sand, gun emplacements, and arrow slits from which defenders could fire at attackers while still enjoying nearly full cover. Spaces in the walls for firing from were called sama (狭間); arrow slits were called yazama (矢狭間), gun emplacements teppōzama (鉄砲狭間) and the rarer, later spaces for cannon were known as taihōzama (大砲狭間). In Japanese castles, the walls' timbers would be left sticking inwards, and planks would be placed over them to provide a surface for archers or gunners to stand on. Other tactics to hinder attackers' approaches to the walls included bamboo spikes planted into the ground at a diagonal, or the use of felled trees, their branches facing outwards and presenting an obstacle to an approaching army. Many castles also had trapdoors built into their towers, and some even suspended logs from ropes, to drop on attackers.

Japanese castles featured massive stone walls and large moats. However, walls were restricted to the castle compound; they were never extended around a jōkamachi (城下町, castle town), and only very rarely were built along borders. This comes from Japan's long history of not fearing invasion. Tile-roofed buildings, constructed from plaster over skeletons of wooden beams, lay within the walls, and in later castles, some of these structures would be placed atop smaller stone-covered mounds. Sometimes a small portion of a building would be constructed of stone, providing a space to store and contain gunpowder.

Though the area inside the walls could be quite large, it did not encompass fields or peasants' homes, and the vast majority of commoners lived outside the castle walls. Samurai lived almost exclusively within the compound, those of higher rank living closer to the daimyō's central keep. In some larger castles, such as Himeji, a secondary inner moat was constructed between this more central area of residences and the outer section where lower-ranking samurai kept their residences. Only a very few commoners, those directly in the employ and service of the daimyō or his retainers, lived within the walls, and they were often designated portions of the compound to live in, according to their occupation, for purposes of administrative efficiency.

The primary method of defense lay in the arrangement of the baileys, called maru (丸) or kuruwa (曲輪 or 郭). Maru, meaning 'round' or 'circle' in most contexts, here refers to sections of the castle, separated by courtyards. Some castles were arranged in concentric circles, each maru lying within the last, while others lay their maru in a row; most used some combination of these two layouts. Since most Japanese castles were built atop a mountain or hill, the topography of the location determined the layout of the maru.

The most central bailey, containing the keep, was called honmaru (本丸), and the second and third were called ni-no-maru (二の丸) and san-no-maru (三の丸) respectively. These areas contained the main tower and residence of the daimyō, the storerooms (kura 蔵 or 倉), and the living quarters of the garrison. Larger castles would have additional encircling sections, called soto-guruwa or sōguruwa. While maru (丸) most literally translates simply to "round" or "circle", kuruwa denotes an area enclosed by earthworks or other walls, and was a term also used to denote the enclosed red-light districts such as the Yoshiwara during the Edo period. As it relates to castles, most castles had three maru, main baileys, which could be called kuruwa; additional areas beyond this would be called gaikaku (外廓), or "kuruwa that are outside". At many castles still standing today in Japan, only the honmaru remains.

A complex system of a great many gates and courtyards leading up to the central keep serves as one of the key defensive elements. This was very carefully arranged to impede an invading army and to allow fallen outer portions of the compound to be regained with relative ease by the garrisons of the inner portion. Since sieges rarely involved the wholesale destruction of walls, castle designers and defenders could anticipate the ways in which an invading army would move through the compound, from one gate to another. As an invading army passed through the outer rings of the Himeji compound, it would find itself directly under windows from which rocks, hot sand, or other things could be dropped, and also in a position that made them easy shots for archers in the castle's towers. Gates were often placed at tight corners, forcing a bottleneck effect upon the invading force, or even simply at right angles within a square courtyard. Passageways would often lead to blind alleys, and the layout would often prevent visitors (or invaders) from being able to see ahead to where different passages might lead. All in all, these measures made it impossible to enter a castle and travel straight to the keep. Invading armies would be forced to travel around and around the complex, more or less in a spiral, gradually approaching the center, all while the defenders prepared for battle, and rained down arrows and worse upon the attackers.

Castles were rarely forcibly invaded, however. It was considered more honorable, and more appropriate, for a defender's army to sally forth from the castle to confront his attackers. When this did not happen, sieges were most often performed not with siege weapons or forced entry, but by surrounding the enemy castle and denying food, water, or other supplies to the fortress. As this tactic could often take months or even years to see results, the besieging army sometimes even built their own castle or fortress nearby. This being the case, the castle was less a defensive fortress than a symbol of defensive capacity with which to impress or discourage the enemy. It also served as the lord's residence, a center of authority and governance, and in various ways a similar function to military barracks.

The castle keep, usually three to five stories tall, is known as the tenshukaku (天守閣), and may be linked to a number of smaller buildings of two or three stories. Some castles, notably Azuchi, had keeps of as many as seven stories. The tallest and most elaborate building in the complex, and often also the largest. The number of stories and building layout as perceived from outside the keep rarely corresponds to the internal layout; for example, what appeared to be the third story from outside may have in fact been the fourth. This must have helped to confuse attackers, preventing them from knowing which story or which window to attack, and likely disorienting the attacker somewhat once he made his way in through a window. During peacetime, the keep mainly functioned as a warehouse to store weapons and armor.

The least militarily equipped of the castle buildings, the keep was defended by the walls and towers, and its ornamental role was never ignored; few buildings in Japan, least of all castle keeps, were ever built with attention to function purely over artistic and architectural form. Keeps were meant to be impressive not only in their size and in implying military might, but also in their beauty and the implication of a daimyō's wealth. Though obviously well within the general sphere of Japanese architecture, much of the aesthetics and design of the castle was quite distinct from styles or influences seen in Shintō shrines, Buddhist temples, or Japanese homes. The intricate gables and windows are a fine example of this.

However, the keep was not the primary residence of the lord; there were separate palace buildings, known as goten (御殿) to serve that purpose. Authentic surviving goten are even rarer than authentic surviving tenshukaku. A famous example on an authentic goten is the Ninomaru Goten of Nijo Castle in Kyoto, which served as the Shogun's residence whenever he was visiting Kyoto. It was built with "nightingale floors" that were deliberately designed to squeak when somebody walked on them so the Shogun and his retainers could be warned of any potential assassins.

Palisades lined the top of the castle's walls, and patches of trees, usually pines, symbolic of eternity or immortality, were planted along them. These served the dual purpose of adding natural beautiful scenery to a daimyō's home, representing part of his garden, and also obscuring the insides of the castle compound from spies or scouts.

A variety of towers or turrets, called yagura (櫓), placed at the corners of the walls, over the gates, or in other positions, served a number of purposes. Though some were used for the obvious defensive purposes, and as watchtowers, others served as water towers or for moon-viewing. As the residences of wealthy and powerful lords, towers for moon-viewing, balconies for taking in the scenery, tea rooms and gardens proliferated. These were by no means solely martial structures, but many elements served dual purposes. Gardens and orchards, for example, though primarily simply for the purpose of adding beauty and a degree of luxuriousness to the lord's residence, could also provide water and fruit in case of supplies running down due to siege, as well as wood for a variety of purposes.