First of all, in this course, computer memory is used to refer to the RAM of the computer, not the storage devices.
RAM stands for Random Access Memory. Unlike the memory inside storage devices, for example, Hard Drive Disks (HDD) and Solid State Drives (SSD), the computer must be running for the RAM to be able to hold data. Once the computer is closed, all the data stored inside the RAM is lost. It is therefore a temporary memory. The reason the RAM is used is that it is way faster than storage devices (To read/write from/to) and the amount of write operations executed on it is not limited (Unlike with SDDs).
The computer memory (And storage devices too actually) is composed of bits. A bit is the smallest unit of the memory. It is a binary digit so it may only represent two values (0 or 1). That is not much, but with two bits we can represent 4 different values and with 8 bits (An octet) we can represent 256 different values. The memory is actually a sequence of bits.
The smallest amount of memory a computer can directly work with is called a byte. Nowadays, a byte is almost always made of 8 bits (An octet), but this is not a guarantee. So even though the memory is composed of bits, we can only access them in groups of the size of a byte. Therefore, the computer memory (And the memory of storage devices) can be seen as a sequence of bytes (A sequence of group of bits of the size of a byte).
Each byte in the RAM is assigned with an address. The first byte has the address 1, the second 2, the third 3 and so on. The address 0 is an invalid address.
For storage devices, it is a bit different. Those devices are divided into sectors that are made of many bytes (Example, 512 bytes) and we can not read and write individual bytes from/to them. We must read and write whole sectors. When we read data from a storage device, the sectors read are saved into the RAM. When we write data to a storage device, the data is copied from the RAM.