NET 226 - Designing Internetwork Solutions

Chapter 5, Designing a Network Topology, part 1


This lesson discusses designing a network from the perspective of its components and connections. Objectives important to this lesson:

  1. Hierarchical design
  2. Flat design vs. hierarchical design
  3. Mesh vs. hierarchical-mesh
  4. Redundant design
  5. Modular design
  6. Campus network and spanning tree
  7. Virtual LANs

Chapter 5


The chapter presents a discussion of four major concepts that relate to topology. The text defines a topology as a map of a network that shows its segments, its interconnection points, and its user communities. This kind of map was discussed in the last chapter as a tool for understanding a network. This chapter looks at a topology as a tool for planning and designing a network. A Cisco Learning article concerning this topic can be reached with this link.

The text makes a point on page 121 that a network that grows without a plan presents problems to its administrators and users. The author calls this a fur-ball network, but she does not explain the metaphor. It may be best if we don't dwell on it.

Hierarchical design

The first of topology topic discussed is hierarchical network design. The illustration on page 120 defines this topology as having three parts:

  • core layer - routers and switches optimized to carry lots of data and to have high availability, it acts as the backbone of the network, and carries data between sites
  • distribution layer - routers and switches with lower capacity than those in the core layer that implement policies, and that connect as a between the access layer and the core layer; they translate protocols as needed between the other two layers
  • access layer - the layer of switches and wireless access points through which users connect their stations to the network, as well as the edge routers that connect LANs to the distribution layer

Back to page 121, the author explains that an unplanned network may grow by adding more and more switches, which can produce a network with only one broadcast domain. This means that any broadcast not only reaches every host on the network, it interrupts each host's processor, wasting time on a message that does not concern most of them. A hierarchical design avoids this problem by producing subnets that are separate broadcast domains. Each subnet is a separate module in the network, and more may be added without causing needless traffic through existing subnets. This design also minimizes traffic from one router to another, avoiding the traffic that would be caused by all routers talking to each other constantly.

Flat design vs. hierarchical design

While hierarchical design includes a modularity that allows for easy expansion, the author tells us that a network that is small and that is meant to remain small may be fine with a flat design. The main drawback the author points out on page 122 is that troubleshooting may require looking at the entire network. In a flat design, all hosts readily interact with every other host.

In the illustrations on page 123, the text shows us two WAN topologies. In the first, routers at four locations are linked in a flat ring. Each router connects to two others, providing a choice of two routes to any of the other routers. A route to another router may be one, two, or three hops. Adding more routers to the ring will increase the number of hops to reach other routers in this flat example.

In the second illustration, we see six routers, two of which are in Medford. They are elevated to an upper layer. They serve as gateways to other networks. Each of the remaining four routers are on a lower layer. They are directly connected to each of the two gateways in Medford, which places the lower level routers one hop away from each Medford router. To reach another bottom layer network, a router only has to contact a Medford router (one hop), then connect to the desired router, a total of two hops. Every route in this design will be one or two hops, which is more efficient than three. The number of hops to another network will not change if we add more networks in either layer.

Netgear HubThe author continues her discussion of flat networks, explaining that hubs were used in flat networks in the early and mid 1990s, but they became troublesome as networks grew. Every device plugged into a hub is in contention with every other device for network access. Bridging from one hub to another just makes the problem worse. The problem was reduced when OSI Layer 2 switches were introduced, which notice the MAC address of each attached device, pass traffic to the necessary port, and do not pass traffic to unnecessary ports. All devices attached to connected switches are still on one broadcast domain, but adding a layer of routers reduces broadcast traffic, because routers typically do not forward broadcasts from one network to another. The hierarchical design of a subnet for each switch or for groups of switches, reduces the problem.

Mesh vs. hierarchical-mesh

The illustrations on the bottom of page 124 each show five routers. In the first illustration, the routers are connected in a ring (a pentagon, actually), each connecting to two other routers. Three of those routers are also connected to a third router. This provides multiple routes for some of the routers to select, which makes this a partial mesh. In the second illustration, every router in the pentagonal ring is directly connected to every other router, which makes this a full mesh. The redundancy of these connections makes the network more resilient and reliable, but there is a cost. The formula at the top of page 125 tells us that for a network of n routers, we would need to make
(n * (n - 1))/ 2
connections for a full mesh of those devices. This runs into a lot of cable and a lot of NICs. The text proposes that a better design is the compromise of adding some redundancy to the connections in a hierarchy, as shown on the top of page 126. It is the same design we saw on page 123, but it has been expanded to include a new layer. Each router in the bottom layer is still connected to two other routers, but the routers in the uppermost layer are connected to three routers, and each router in the middle layer is connected to four or more routers. This design give a lot of redundancy but less than full redundancy.

Page 126 also shows an illustration of a more common hub and spoke hierarchy. Think of the central office of this company as the center (hub) of a wheel, and each satellite office as the end of a spoke of that wheel. This common design does not use any redundancy, so each connection is an important one to someone. Note: the use of the word hub in this case has nothing to do with the concept of network hubs. It's nice to use the same word for two things, isn't it?

This takes us to an expanded discussion of the hierarchical model. Some of the information here is summarized in the bullet list above. The author also shows us two common mistakes made in hierarchical networks on page 129. In the first example, on the left side of the illustration, an admin has connected a new router to another router on the access layer. This creates a chain, a new layer in the network. That new router should have been connected to a router on the distribution layer, to maintain what the author calls the network diameter. In the second example, on the right side of the illustration, we see a new router has been connected to two routers on the access layer. The author calls this a back door error. It leads to additional routes that the routers must communicate to each other, and to a loop in that part of the network. Again, this new router should have been properly connected to a router in the distribution layer to make it a proper member of the network. These errors could be made by connecting a router, a switch, or a hub incorrectly.

Redundant design

As we have already seen in the mesh examples, redundancy on a network can provide reliable connections when one component goes down. The author explains that we should consider redundancy for any critical network component, not just linkages through routers. Well engineered redundancy increases reliability, but it increases expense as well, so the text cautions us to consider the client's budget when planning redundancy.

Redundant routes are discussed on page 131 under the topic of backup paths. (This is about having a second or third choice of a route, not about backup of data.) When routers select routes, they typically select the best on based on some metric, so you can optimize your network performance by supplying metric data that your routers understand. This can lead to the author's desired state of automatic failover, which is preferred over a manual process to change available routes.

The author also discusses load sharing, which is sometimes called load balancing. With regard to networking, you need to know that load sharing splits a data stream across multiple routes, and reassembles them on the far end. We expect this to happen for every packet in an IP network, so this should not be a new concept.

Modular design

The author suggests that we can also consider creating a network as a set of modules that are meant to interact in proven ways. She recommends a structure that Cisco calls SAFE. Its twelve major components are described on pages 134 and 135. The remainder of the chapter discusses several of these modules.

Campus network

The major topic under the author's discussion of our campus (main) network is spanning tree protocol. This protocol was developed to deal with a problem that arises in networks that have redundant paths to a given destination. Weren't we told we should have redundant paths? Yes, we were, and yes, we should. So, what's the problem, since the author doesn't seem to want to describe it? Well, she does tells us that the rules for this kind of situation were invented when everyone used bridges to connect or divide network segments, so she is using the word "bridge" to mean both bridges and layer 2 switches. That's good to know. Now, the problem:

The bottom line is that bridges and switches are (mostly) layer 2 devices. They work with MAC addresses. When there are two (or more) paths across switches from one segment to another, and the switches on those paths both forward packets, each switch sees the packet forwarded by the other switch, and the devices get confused about which segment the addresses live on. This is bad.

Enter the spanning tree protocol, like a hero to solve our problem. The protocol says that if there are two ways to cross over to another segment, the devices will determine which device has the best connection, and use the route across that device. The process of choosing the best path involves each device sending information about itself and its connections to every other such device. This process is carried out every time a new path is added or lost, and the process of defining the preferred paths is called convergence. The text tells us that Rapid Spanning Tree Protocol (RSTP) allows devices can reach convergence in "a few hundred milliseconds". (page 137)

Every port on a switch can be in one of several states. Three are listed on page 137:

  • Discarding - a port in this state does not learn new MAC addresses or forward frames
  • Learning - a port in this state is learning MAC addresses and associated ports, and is writing information in the MAC address table of its switch/bridge
  • Forwarding - a port in this state is doing the things in the learning mode, and it is also forwarding frames based on information in the MAC address table

Spanning tree protocol also causes bridges to elect a root bridge. Remember that information the bridges/switches send about themselves? It includes their IDs, composed of their MAC address and a code assigned by an admin. The bridge with the lowest ID is automatically the root bridge. Note that the admin can rig the election by assigning a low code to a favorite bridge. If all codes were the same, the bridge with the lowest MAC address would be the root bridge.

Another feature of the spanning tree protocol is that it calls for each device to participate in sharing its information regularly. If a designated bridge (the one being used for a segment) has failed to communicate recently, the root bridge can call for a new election, which will result in choosing a new designated bridge for a segment whose designated bridge has gone down.

Virtual LANs

This section of the chapter contains an unrelated nugget of information. The author defines a bandwidth domain as any set of devices that share bandwidth or compete for access to it. In a classic wired Ethernet, there was one bandwidth domain, because all devices on that LAN competed for access with each other. In an Ethernet with switches, the text tells us that each device that is wired to a switch is on its own bandwidth domain, but this is a little specious, since the there is no point in communicating only with the switch.

The text also defines a broadcast domain as the set of devices that can receive each other's broadcasts frames. This is better definition than we usually see. We are reminded (or told, if we did not know) that the broadcast address for layer 2 is a MAC address that is all Fs.

The text turns a corner and steers toward VLANs. Users anywhere on your network can be made members of a common Virtual LAN, which lets them communicate as easily as if they were on the same LAN. This was the original use of VLANs. They are not often used for this purpose any longer. Usually, VLANs are used, as the text says, to make a large switch act as though it was really several switches, so that it can be used to separate groups of ports into different VLANs. This has the benefit of having each VLAN act as a separate broadcast domain, minimizing broadcast intrusions for all devices plugged into ports on that switch. A virtual router on the switch connects the separate VLANs the same way a real router would.

As you might imagine from the description of VLAN users being anywhere in your network, a VLAN can exist on specific ports of multiple switches. When this is done, the connections between the switches that contain the parts of a VLAN are called trunks or trunk links. Frames traveling from one such switch to another are given a header identifying the VLAN it belongs to. The header is called a VLAN tag.

Week 4 Assignment: Chapter 5

  • From Chapter 5:
    • Study the chapter.
    • Design Scenario: page 165, Questions 1 - 4
  • Read Chapter 13