It would be nice to think that you could just go and buy any computer, install a bit of software on it and start using it to record and create music. This idea isn't wrong, but there some important details that it misses.

Any computer that you can buy today (since somewhere around the end of 2012) is capable of recording and processing a lot of audio data. It will come with a builtin audio interface that can accept inputs from microphones or electrical instruments. It will have a disk with a huge amount of space for storing audio files.

When you are recording, editing and mixing music, you generally want to work with very little latency between the time that a sound is generated and when you can hear it. When the audio signal flows through a computer, that means that the computer has to be able to receive the signal, process it and send it back out again as fast as possible.
And that is where it becomes very important what computer system you have, because it is absolutely not the case that any computer can do this job well.

Routing audio through a computer will always cause some delay, but if it is small, you will generally never notice it. There are also ways to work in which the delay does not matter at all (for example, not sending the output from the computer to speakers).

The latency that you want for working with digital audio is typically in the 1–5 ms range. For comparison, if you are sitting 1 m (3 ft) from your speakers, the time the sound takes to reach your ears is about 3 ms. Any modern computer can limit the delay to 100 ms. Most can keep it under 50 ms. Many will be able to get down to 10 ms without too much effort. If you try to reduce the delay on a computer that cannot meet your goal, you will get clicks and glitches in the audio, which is clearly extremely undesirable.

Hardware-related Considerations

Video interface: Poorly engineered video interfaces (and/or their device drivers) can "steal" computer resources for a long time, preventing the audio interface from keeping up with the flow of data
Wireless interface: Poorly engineered wireless networking interfaces (and/or their device drivers) can also block the audio interface from keeping up with the flow of data
USB ports: If you are using an audio interface connected via USB, and sometimes even if you are not, the precise configuration of your system's USB ports can make a big difference. There are many cases where plugging the interface into one port will work, but using different USB port results in much worse performance. This has been seen even on Apple systems.
Internal USB Hubs: Ideally, you'd like your USB ports to all connect directly to the main bus inside the computer. Some laptops (and possibly some desktop systems) come wired with an internal USB hub between the ports and the system bus, which can then cause problems for various kinds of external USB devices, including some models of audio interfaces. It is very difficult to discover whether this is true or not, without simplying trying it out.
CPU speed control: Handling audio with low latency requires that your processor keeps running at its highest speed at all times. Many portable systems try to regulate processor speed in order to save power—for low latency audio, you want this totally disabled, either in the BIOS or at the OS level.
Excessive Interrupt Sharing: If your audio interface is forced by your computer to share an interrupt line (basically a way to tell the CPU that something needs its attention) with too many, or the wrong, other devices, this can also prevent the audio interface from keeping up with the flow of data. In laptops it is generally impossible to do anything about this. In many desktop systems, it is possible at the BIOS level to reassign interrupts to work around the problem.
SMIs: SMIs are interrupts sent by the motherboard to tell the computer about the state of various hardware. They cannot safely be disabled, but they can also take a relatively long time to process. It is better to have a motherboard which never sends SMIs at all—this is also a requirement for realtime stock trading systems, which have similar issues with latency.
Hyperthreading: This technology is becoming less common as actual multi-core CPUs become the norm, but it still exists and is generally not good for realtime performance. Sometimes you can disable this in the BIOS, sometimes you cannot. A processor that uses hyperthreading will be less stable in very low latency situations than one without.
Excessive vibration: This doesn't affect the flow of data to/from the audio interface, but it can cause the flow of data to/from your disk storage to become much slower. If you are going to use a computer in an environment with loud live sound (specifically, high bass volume), make sure to place it so that the disk is not subject to noticeable vibration. The vibrations will physically displace the head-write heads of disk, and the resulting errors will force a retry of the reading from the disk. Retrying over and over massively reduces the rate at which data can be read from the disk. Avoid this.