阅读说明：本技术 专注于延时的读取操作和专注于带宽的读取操作之间的动态改变 (Dynamic change between read operations devoted to latency and read operations devoted to bandwidth ) 是由 S·哈利利 Z·格林菲尔德 S·贾亚钱德兰 R·J·小罗耶 D·帕特尔 于 2020-06-24 设计创作，主要内容包括：多级存储器子系统包括持久性存储器设备,该持久性存储器设备可以顺序地或随机地访问数据块以改善读取延时,或者可以预取数据区块以改善读取带宽。介质控制器在顺序地或随机地访问数据块的第一读取模式与预取数据区块的第二读取模式之间动态切换。介质控制器基于命令队列中未决的读取命令的数量来在第一读取模式和第二读取模式之间切换。(The multi-level memory subsystem includes a persistent memory device that can sequentially or randomly access data blocks to improve read latency or can prefetch data blocks to improve read bandwidth. The media controller dynamically switches between a first read mode in which the data blocks are accessed sequentially or randomly and a second read mode in which the data blocks are pre-fetched. The media controller switches between the first read mode and the second read mode based on a number of read commands pending in the command queue.)

1. A memory device, comprising:

a persistent storage media accessible as a block of data, the block of data having a size at least equal to an access size of the volatile media cache; and

a media controller to control access to the persistent storage media, the media controller to dynamically switch between a first read mode and a second read mode based on a number of pending read commands in a command queue, the first read mode to access only data blocks addressed in the pending commands and the second read mode to access data blocks addressed in the pending commands and to pre-fetch consecutive data blocks of the data blocks.

2. The memory device of claim 1, wherein the persistent storage medium comprises a cross-point memory medium having multiple layers of memory cells.

3. The memory device of claim 1, wherein the media controller comprises a controller on a Printed Circuit Board (PCB) module having a plurality of persistent media dies.

4. The memory device of claim 1, wherein the media controller is to preset to the first read mode and switch to the second read mode in response to a threshold number of read commands pending in the command queue.

5. The memory device of claim 4, further comprising:

a register to store a programmable value to set the threshold number for triggering switching from the first read mode to the second read mode.

6. The memory device of claim 4, wherein the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands.

7. The memory device of claim 1, further comprising a prefetch buffer to store prefetched data blocks.

8. The memory device of claim 7, wherein in the second read mode, in response to a read command addressed to a data block within the data block that has been prefetched, the media controller is to return the data block from the prefetch buffer.

9. The memory device of claim 7, wherein in the second read mode, in response to a read command addressed to a data block within the data block, the media controller is to queue the read command to wait for a prefetch of the data block.

10. A system having multi-level memory, comprising:

a volatile memory device including a volatile medium as a near memory for the processor; and

a persistent memory device comprising a persistent storage medium as a far memory of the processor, the persistent memory device comprising:

a persistent storage medium as a remote memory of the processor, the persistent storage medium accessible as a data block, the data block having a size at least equal to an access size of the volatile medium; and

a media controller to control access to the persistent storage media, the media controller to dynamically switch between a first read mode and a second read mode based on a number of pending read commands in a command queue, the first read mode to access only data blocks addressed in the pending commands and the second read mode to access data blocks addressed in the pending commands and to pre-fetch consecutive data blocks of the data blocks.

11. The system of claim 10, wherein the media controller is to preset to the first read mode and switch to the second read mode in response to a threshold number of read commands pending in the command queue.

12. The system of claim 11, the persistent memory further comprising:

a register to store a programmable value to set the threshold number to trigger a switch from the first read mode to the second read mode, wherein the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands.

13. The system of claim 11, wherein the persistent memory comprises a hardcoded value to set the threshold number for triggering a switch from the first read mode to the second read mode, wherein the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands.

14. The system of claim 10, the persistent memory further comprising:

a prefetch buffer for storing prefetched data blocks.

15. The system of claim 14, wherein in the second read mode, in response to a read command addressed to a data block within the data block that has been prefetched, the media controller is to return the data block from the prefetch buffer; and the number of the first and second electrodes,

wherein, in the second read mode, in response to a read command addressed to a data block within the data block, the media controller is to queue the read command to await a prefetch of the data block.

16. The system of claim 10, further comprising one or more of:

a host processor device coupled to the volatile memory device and the persistent memory device;

a display communicatively coupled to the host processor;

a network interface communicatively coupled to the host processor; or

A battery for powering the system.

17. A method for storing data, comprising:

receiving a read command from a host controller;

determining whether a number of pending read commands in a command queue meets or exceeds a threshold number of read commands in the command queue; and

dynamically switching from a first read mode to a second read mode in response to determining that the number of read commands meets or exceeds the threshold number, the first read mode for accessing only data blocks addressed in pending commands and the second read mode for accessing data blocks addressed in pending commands and pre-fetching consecutive data blocks of a data block.

18. The method of claim 17, wherein dynamically switching from the first read mode to the second read mode comprises:

presetting to the first reading mode; and

switching to the second read mode in response to a threshold number of read commands pending in the command queue.

19. The method of claim 17, further comprising:

storing a programmable value in a register to set the threshold number for triggering a switch from the first read mode to the second read mode, wherein switching from the second read mode to the first read mode comprises switching in response to the command queue having no pending read commands.

20. The method of claim 17, further comprising: the prefetched data block is stored in a prefetch buffer.

21. The method of claim 20, wherein in the second read mode, responsive to a read command addressed to a data block within the data block that has been prefetched, returning the data block from the prefetch buffer; and the number of the first and second electrodes,

in response to a read command addressed to a data block within the data block, queuing the read command to await a prefetch of the data block.

22. An article of manufacture comprising a computer-readable storage medium having content stored thereon, which when executed, cause a device to perform operations comprising:

receiving a read command from a host controller;

determining whether a number of pending read commands in a command queue meets or exceeds a threshold number of read commands in the command queue; and

dynamically switching from a first read mode to a second read mode in response to determining that the number of read commands meets or exceeds the threshold number, the first read mode for accessing only data blocks addressed in pending commands and the second read mode for accessing data blocks addressed in pending commands and pre-fetching consecutive data blocks of a data block.

23. The article of manufacture of claim 22, wherein dynamically switching from the first read mode to the second read mode comprises:

applying the first reading mode in a preset manner; and

switching to the second read mode in response to a threshold number of read commands pending in the command queue.

24. The article of manufacture of claim 22, wherein the operations further comprise:

storing a programmable value in a register to set the threshold number for triggering a switch from the first read mode to the second read mode, wherein switching from the second read mode to the first read mode comprises switching in response to the command queue having no pending read commands.

25. The article of manufacture of claim 22, wherein the operations further comprise storing the prefetched data chunk in a prefetch buffer;

wherein in the second read mode, in response to a read command addressed to a data block within the data block that has been prefetched, the data block is returned from the prefetch buffer; and the number of the first and second electrodes,

in response to a read command addressed to a data block within the data block, queuing the read command to await a prefetch of the data block.

Technical Field

In general, description is made with respect to memory reads, and more particularly with respect to dynamically switching between a read mode that focuses on latency and a read mode that focuses on bandwidth.

Background

When a host system issues a read command to the memory or cache layer, there is a latency associated with the execution and completion of the command. The latency of the command is fairly deterministic when no other commands are pending. However, as the number of outstanding or pending commands from the host increases, latency is no longer as deterministic. Bandwidth at the storage medium will affect the latency of all command completions except commands sent when other commands are not pending.

In systems sensitive to read latency, media limitations have a significant performance impact. Therefore, systems that are sensitive to read latency typically optimize media access for read latency. However, bandwidth also translates into latency when there are multiple outstanding reads, and the system can optimize media access for read latency or bandwidth.

Drawings

The following description includes a discussion of the figures with illustrations given by way of example of embodiments. The drawings should be understood by way of example and not by way of limitation. As used herein, reference to one or more examples is understood to describe a particular feature, structure, or characteristic included in at least one embodiment of the invention. The appearances of phrases such as "in one example" or "in an alternative example" in this document provide examples of embodiments of the invention and are not necessarily all referring to the same embodiment. However, they are also not necessarily mutually exclusive.

FIG. 1 is a block diagram of an example of a system that can access a storage medium that focuses on latency or on bandwidth depending on the number of pending read requests.

FIG. 2A is a timing diagram of an example of a command sequence that addresses latency for a queue depth of 1 read.

FIG. 2B is a timing diagram of an example of a command sequence that focuses on a bandwidth of reads greater than queue depth 1.

FIG. 2C is a timing diagram of an example of a command sequence that focuses on bandwidth with a queue depth of prefetched 1-reads after servicing a particular read.

Fig. 2D is a timing diagram of an example of a command sequence to switch between latency mode and bandwidth mode based on the number of pending commands.

FIG. 3 is a swim lane flow diagram of an example of switching between a latency mode and a bandwidth mode. The diagram is divided into three parts: FIG. 3-1, FIG. 3-2 and FIG. 3-3.

FIG. 4 is a block diagram of an example of a system having a memory subsystem with near memory and far memory, and with an integrated near memory controller and an integrated far memory controller.

FIG. 5 is a block diagram of an example of a non-volatile storage system with access mode control for varying control based on a number of queued read requests.

FIG. 6 is a block diagram of an example of a computing system in which switching between read modes based on queue depth may be implemented.

Fig. 7 is a block diagram of an example of a mobile device in which switching between read modes based on queue depth may be implemented.

The following is a description of certain details and embodiments, including non-limiting descriptions of the drawings, which may depict some or all examples, as well as other potential embodiments.

Detailed Description

As described herein, a multi-level memory subsystem includes a persistent memory device that can sequentially access data blocks to improve read latency, or can prefetch data blocks to improve read bandwidth. In general, the system may optimize media access of the memory to improve read latency or improve bandwidth utilization. As used herein, "optimizing" does not necessarily refer to absolute optimization, but rather a configuration operation to improve the desired result. Thus, optimizing read latency refers to configuring the system to attempt to provide the best read latency possible in the system. Similarly, optimizing bandwidth may refer to configuring operations to attempt to achieve the best bandwidth utilization possible for the system.

Instead of being optimized for read latency or bandwidth, the media controller may switch between a latency mode optimized for read latency and a bandwidth mode optimized for bandwidth. Access to the medium may be performed in slices or blocks, or in sectors or data blocks. When reading a data block, the reading may be random, leading to the possibility of medium access collisions, and thus to delays between random accesses. No access conflict occurs when reading all blocks in a block, so the reading of the entire slice can be performed in a manner that improves bandwidth utilization. Thus, optimizing for a particular workload type may improve overall bandwidth and read latency.

The media controller switches based on the number of pending commands. Thus, the media controller dynamically switches between a first read mode in which the data blocks are sequentially accessed and a second read mode in which the data blocks are pre-fetched. Sequential access data blocks do not include prefetching of data and, therefore, read latency will be reduced since the system can simply service incoming commands. Accessing data using prefetched data blocks having multiple contiguous data blocks allows the media controller to cache data to reduce latency in sending data over the communication signal line, thereby improving bandwidth utilization. In the case of data block access, data blocks are sequentially accessed and transmitted, thereby deteriorating bandwidth utilization. In the case of prefetching, the processing of incoming commands may be delayed to wait for prefetching of other data, thereby worsening the read latency.

The media controller switches between the first read mode and the second read mode based on a number of read commands pending in the command queue. In one example, the trigger between the first read mode and the second read mode is when there is at least one pending command in the command queue. Other implementations may be to trigger a change between read modes by 2, 3, or some other number of queued commands.

FIG. 1 is a block diagram of an example of a system that can access a storage medium that focuses on latency or on bandwidth depending on the number of pending read requests. System 100 represents a system in which host 110 accesses data stored in memory device 120. The memory device 120 includes a medium 124 for storing data, and a medium controller 130 for dynamically switching between a read mode that focuses on read latency and a read mode that focuses on bandwidth utilization.

Host 110 represents a host hardware platform that includes processor 114 to execute an Operating System (OS), an application, or a process that generates requests for data stored on memory device 120. The processor 114 may be or include any type of microprocessor, microcontroller, central processing unit, graphics processing unit, or other device that executes a sequence of commands that may cause a data access request to the memory device 120. In one example, the processor 114 includes multiple cores (i.e., a multi-core processor). Different threads executing on a single processor or a single core may generate different memory access requests.

I/O (input/output) 112 includes hardware elements used to interconnect host 110 to memory device 120. Memory device 120 includes corresponding I/O122. The I/O includes one or more signal lines to enable the host 110 to send commands to the memory device 120. The I/O includes one or more signal lines to enable the memory device 120 to return data to the host 110. In one example, I/O112 and I/O122 include separate Command and Address (CA) signal lines and Data (DQ) signal lines. The width of the data bus may vary depending on the architecture. I/O112 and I/O114 include transmitters and receivers for driving and receiving signals on signal lines, respectively, or transceivers for controlling transmission and reception on signal lines. The interface hardware may be controlled by software and firmware operations to control the timing and operating parameters of the interface.

The host 110 includes a controller 116 that represents control logic for generating commands and scheduling the sending of commands and processing returned data based on operations performed by the processor 114. In one example, the controller 116 is integrated on a common die with the processor 114 (i.e., an integrated controller). In one example, the controller 116 is a separate component from the processor 114. The controller 116 may implement protocol rules to cause the signaling via the I/O. In one example, the controller 116 configures the operation of the transmit and receive components of the I/O. In one example, the controller 116 is or includes a scheduler for scheduling signaling. The controller 116 may generate the illustrated request to the memory device 120, which will provide the illustrated reply in response to the request.

The memory device 120 includes a medium 124 that represents storage space on the memory device. In one example, media 124 represents non-volatile storage media. The medium 124 may be OR include a NAND (non-AND) based nonvolatile storage, NOR (non-OR) based nonvolatile storage, OR a 3DXP (three dimensional cross point) nonvolatile storage that stores data based on the resistance state of a bit cell OR other resistance based nonvolatile storage. Media 124 includes a plurality of addressable storage locations. In one example, the storage locations are addressable on a block basis. In one example, the storage locations are byte addressable (e.g., 3DXP memory). A block or byte includes individual bit cells for storing one or more bits of data (e.g., a multi-level cell device stores multiple bits per cell).

In one example, memory device 120 includes a plurality of media units according to media 124. For example, medium 124 may represent a single chip or die or a plane of storage space. Memory device 120 may include multiple chips or other memory units. In one example, media controller 130 interfaces with all media units. In one example, the memory device 120 includes different media controllers that may individually control access to the storage media according to any of the examples described.

Media controller 130 represents the controller or control logic on memory device 120 for controlling access to media 124. In one example, the storage locations of media 124 may be accessed as individual sectors (e.g., sectors [0:7], identified as SECT 0, SECT 1, … …, SECT 7). In one example, a slice of a storage device includes a plurality of sectors. As shown in system 100, slice 0 includes sectors [0:3], and slice 1 includes sectors [4:7 ]. The slice to sector correspondence may be different than 1:4 as shown.

The media controller 130 receives commands for execution from the host 110, identified as "requests" in the system 100. Each request has an associated command latency, which is a minimum expectation of command completion. The relationship between bandwidth and latency can be described as follows. The first command received after the controller becomes idle has a deterministic latency, while subsequent commands in a series or burst of commands are less deterministic because it is not always known how long it will take to complete the command. The last command of a burst must wait until all previous commands have been completed before completion can be issued. It will be appreciated that the higher the bandwidth, the shorter the time to wait for a subsequent command.

In systems sensitive to read latency, media access limitations may have a significant performance impact, which implies that command processing is optimized for read latency. However, insufficient bandwidth may cause a delay to occur when there are multiple outstanding or pending reads in the command (cmd) buffer 132 (which may also be referred to as a command queue). Media reads may have an optimal read granularity for a certain amount of data, and many reads may be for less than the optimal amount of data. For purposes of this discussion, "slice" refers to the native read bandwidth, or the amount of data that the medium is capable of reading in one unit cycle or clock cycle. A "block" of data refers to an amount of data that is less than a slice; thus, a slice is composed of multiple blocks of data.

In one example, the media 124 requires a significant amount of idle time after reading to avoid address conflicts. An address conflict refers to the situation when a subsequent command addresses the same portion of the medium as a previous command. When a portion of the medium is being accessed for one command, the portion cannot be accessed simultaneously for another command, provided that the control logic cannot address different locations within the portion simultaneously. Idle time may be required before the same address or the portion of the medium in which the address is contained (e.g., up to 1GB of medium for some 3DXP implementations). Increased latency to idle time reduces bandwidth, which results in increased latency for command queue depths greater than 1.

To improve bandwidth, the layout of media 124 may be organized in such a way that sequentially reading an entire slice of the media will not result in media address conflicts. It will be understood that the host address or addresses provided by the host 110 need not be the same as the media address. The media controller 130 controls media addresses, which may be transparent to the host 110. In one example, the medium 124 has a sequential pattern of approximately 2K bytes. In such a configuration, the slices may comprise 2K bytes, where each slice comprises a plurality of 512 byte sectors. In one example, the layout is optimized for accessing slices instead of accessing sectors.

At the media level for this configuration, reading only one sector will be optimized for latency. The memory device 120 will return the requested sectors to the host 110 as quickly as the media controller 130 can read the requested sectors. However, if all host reads are converted to media sector reads, processing delays will result due to media address conflicts. Media address conflicts will translate into reduced bandwidth and hence increased latency for queue depths greater than one. The increased latency will slow down the processor 114 and make the user experience less than optimal.

In one example, memory device 120 comprises a byte-addressable, non-volatile storage medium that can be coupled to a conventional system memory bus. Thus, for example, memory device 120 may share a bus with conventional volatile memory such as DRAM (dynamic random access memory). Memory traffic on a client system may include a 64 byte random access read workload with volatile memory, and a larger sequential access workload, which may be 32 times larger than a 64 byte random access read. In one example, the ability to optimize for read latency or bandwidth may allow for optimization for both types of workloads (64 byte access or 2 kbyte access).

Random read latency is critical for many applications, particularly if the memory device is to be used as an nth level memory in an N-level memory system. For example, a two-level memory system includes a near memory and a far memory, where the near memory has a faster access time and the far memory has a slower access time. The access time may be different because of the device architecture (e.g., different types of devices with different access latencies), different electrical distances from the host, or a combination of electrical distances and device architecture.

Media controller 130 may provide optimizations for both latency and bandwidth for media 124. In one example, the system 100 converts a host read to a media read to eliminate or reduce the impact of media address conflicts on host read bandwidth for both sequential reads and random reads. In one example, the media controller 130 prioritizes delay optimization while the request pressure is low or when there are several pending requests. As one or more command queues fill up and the latency becomes longer, media controller 130 dynamically switches to prioritize bandwidth optimization. In the case of such operation of media controller 130, a secondary benefit of optimizing bandwidth is that latency is also optimized when the command queue is more full.

Referring to configurations in which data on the medium 124 may be viewed as organized into sectors and slices, different read modes may be viewed as sector read modes and slice read modes. A sector read mode may refer to a mode in which the media controller 130 accesses data one sector at a time in response to a request for a particular sector. Slice read mode may refer to a mode in which media controller 130 prefetches data slices in response to a request for a particular sector.

In such an example, when the first read is for only one sector, the latency can be optimized by reading only the requested sector, rather than the entire slice. In general, latency can be optimized by increasing bandwidth utilization when there are multiple outstanding reads. In turn, bandwidth utilization may be optimized by reading the entire data slice, which may reduce address collision latency.

Media controller 130 may dynamically identify when to switch from sector read mode to slice read mode. Based on the control of access to the media 124, a determination is made by the media controller 130 to read the entire slice, regardless of whether the host requests only sectors or the entire slice. Thus, in response to a request for a sector, the media controller may access only the sector or access the entire slice, send the requested data back to the host, and buffer or buffer the remaining data. If the host sends another request for sectors of a slice shortly after the media controller prefetches data, the workload from the host will benefit from latency improvements.

In one example, when there are no outstanding reads, the media controller 130 will be in sector read mode to optimize for latency. The switch from the sector read mode to the slice read mode or the prefetch mode will depend on the number of pending read requests that trigger the switch between modes. Once the number of outstanding reads has reached a programmable threshold, media controller 130 switches to slice reads and prefetches data. In one example, the threshold is a programmable value. For example, the programmable values may be stored in a register device. In one example, the threshold is a fixed value or a hard-coded value. The hard-coded values may be part of the controller firmware, for example, or set to values in a fuse.

In one example, memory device 120 includes registers 134 to store configuration information related to access of media 124 by media controller 130. In one example, registers 134 represent a plurality of registers or storage locations for storing configuration information. In one example, the register 134 is part of the media controller 130. In one example, the register 134 stores a programmable value to set a threshold number that triggers the media controller 130 to switch from one read mode to another read mode. For example, register 134 may include a value set in the configuration or initialization of memory device 120 to switch from a preset read access mode to a different read access mode. More specifically, the register 134 may store a value in the command buffer 132 indicating the number of pending commands that will trigger the media controller 130 to switch from preset read latency optimization to bandwidth optimization. In one example, when the queue is emptied, the media controller 130 switches back to the preset read access mode. In one example, the memory device 120 includes a register 134 to store a value indicating the number of pending commands that will trigger the media controller 130 to switch from the bandwidth optimized read access mode back to the preset read latency optimized read access mode. In one example, the sector read mode or the read mode without prefetching is a preset read mode, and the read mode with prefetching is another mode to switch to in response to the queue depth.

The above description relates to control by media controller 130 to effect dynamic switching from one read mode to another. In alternative examples, the controller 116 may implement the dynamic switching described. Dynamic switching may occur in accordance with the above, wherein the controller 116 issues commands for a sector (or a sub-portion of the media 124) and commands for a slice (or a portion of the media 124 that includes multiple sub-portions) in one read mode. The distinction of sub-portions and portions may be based on unit accesses to memory device 120 and unit pre-fetches to medium 124, respectively.

When the controller 116 effects a change in read mode, the controller will send a different read command to the memory device 120. The controller 116 may include a command buffer or command queue (not explicitly shown) that may be used to determine when to switch read modes. As an example, the controller 116 may switch back in response to receipt of all outstanding requested data.

In one example, media controller 130 (or controller 116) configures system 100 to include a configuration that sets a queue depth that will trigger a change from one read mode to another with reference to the number of pending commands in command buffer 132 (or a similar command buffer of controller 116). In one example, the configuration may include a queue depth for triggering a switch from reading the data block to reading the data block with prefetching, and the controller automatically switches back to reading the data block when the queue is emptied. In one example, the configuration may include a queue depth to trigger a switch from the read block to the read bank, and another queue depth indication may trigger a switch back to the read data block, where the queue is not completely emptied before switching back.

As shown, memory device 120 includes I/O (input/output) buffer 126 as a buffer for transmitting data from media 124 to host 110 through I/O122. I/O buffer 126 represents a buffer used to queue data for transmission to host 110. I/O buffers are used in both mode 1 and mode 2. In one example, I/O buffer 126 is part of I/O122. For discussion purposes, the I/O buffer is specified separately for the two read modes.

As shown, in mode 1, data from the medium 124 is accessed and provided directly to the I/O buffer 126 for transmission to the host 110. For mode 2, data is first accessed and placed in prefetch cache 128 (which may also be referred to as a prefetch buffer) and then provided to I/O buffer 126 in response to a request from host 110. It will be appreciated that not all of the data stored in prefetch cache 128 is sent to host 110. In one example, in response to a simultaneous request operating in the prefetch access mode (mode 2), the media controller 130 will first look for the requested data in the prefetch cache 128. If there is a cache hit, the data may be moved to I/O buffer 126. If there is a cache miss, the media controller 130 will prefetch data associated with the requested data and store the prefetched data in the prefetch cache 128. The requested data will be sent to the host 110 while other data will remain cached until requested or until evicted in accordance with a cache eviction operation implemented by the media controller 130.

In one example, for a cache hit in prefetch cache 128, other commands may have preceded the command for sending data back to host 110. For example, I/O buffer 126 may queue multiple data blocks for transmission in response to a previous command. When a command causes a cache hit, the command may be queued for I/O output and the media controller 130 need not perform additional access operations on the media 124 because the data is already cached in the prefetch cache 128.

In one example, memory device 120 includes circuitry 140, which represents circuitry that enables switching between read modes based on queue depth. In one example, the circuit 140 is a circuit integrated onto a component of the memory device 120. In one example, the circuit 140 is implemented as an Application Specific Integrated Circuit (ASIC). Such circuitry may be integrated onto the assembly or packaged separately. Thus, the circuit 140 may be provided as a separate chip or circuit device to be integrated into the memory device 120, as a chip to be integrated into a system-on-a-chip, or into the silicon of another component of the memory device 120.

FIG. 2A is a timing diagram of an example of a command sequence that addresses the latency of the queue depth for 1 read. Diagram 210 represents commands sent from a host and responded to from a memory device. Diagram 210 represents commands from a host and operations at a media controller and media optimized based only on read latency.

CMD (command) 212 represents a command or read request sent by the host. Each element shown represents a separate command for a separate data block. These commands are indicated with shading to indicate what portion of the data the block is associated with. The legends indicate slice 0 of the shaded (a) block, slice 1 of the shaded (b) block, slice 2 of the shaded (c) block, and slice 3 of the shaded (d) block. The diagram 210 represents a situation in which there are commands sent for blocks of different portions or slices that are interleaved with each other.

For example, CMD 212 represents a request for a block of data from slice 0, followed by a request for data from slice 1. After a period of time, the host sends three consecutive requests for data blocks from slice 0. After a while, the host sends three consecutive requests for data blocks from slice 1, followed by a request for data from slice 2. The host then sends a command for the block of slice 2, followed by a command for the block of slice 3.

CMD 214 represents command encoding from the controller to the storage medium. The host generates the command shown in CMD 212 to request a particular block of data. CMD 214 represents a command generated by the media controller for accessing the media. When the controller is optimized for read latency only, the controller generates a separate command for each data block in response to each command from the host.

Data 216 represents operations at the medium for providing to the buffer. In case of pure read latency optimization, the buffer may be an output buffer to be sent to the host. Data 218 represents operations at an output driver between the memory device and the host. It will be seen that the difference between data 216 and data 218 is a short delay to allow the data to pass from the buffer to the output driver. Both the data in the buffer and the data to the host appear as a single data block, mirroring the request from the host in CMD 212.

FIG. 2B is a timing diagram of an example of a bandwidth-focused command sequence for reads greater than queue depth 1. Diagram 220 represents commands sent from the host and responded to from the memory device. The diagram 220 represents commands from the host and operations at the media controller and media based solely on bandwidth optimization. More specifically, in response to a host read of a sector, the media controller prefetches or reads the entire slice. This approach is optimized for bandwidth and sacrifices read latency for a queue depth of 1(QD1) read.

CMD222 represents a command or read request sent by the host. Each element shown represents a separate command for a separate data block. These commands are indicated with shading to indicate what portion of the data the block is associated with. The legends indicate slice 0 of the shaded (a) block, slice 1 of the shaded (b) block, slice 2 of the shaded (c) block, and slice 3 of the shaded (d) block. The diagram 220 represents a situation in which there are individual blocks or sectors sending commands for different portions or slices that are interleaved with each other.

For example, CMD222 represents a request for a block of data from slice 0, followed by a request for data from slice 1. After a period of time, the host sends three consecutive requests for data blocks from slice 0. After a while, the host sends three consecutive requests for data blocks from slice 1, followed by a request for data from slice 2. The host then sends a command for the block of slice 2, followed by a command for the block of slice 3.

CMD 224 represents the command encoding from the controller to the storage medium. The host generates the command shown in command CMD222 to request a particular block of data. CMD 224 represents a command generated by the media controller for accessing the media. When the controller is optimized for bandwidth only, the controller generates commands for the entire data slice in response to each command from the host. It will be understood that CMD 224 shows only a single command to the media to read an entire slice of data because the entire slice is prefetched.

Data 226 represents operations at the medium for providing to the buffer. In the case of bandwidth optimization, the buffer may be a prefetch cache that will retain data until it is transferred to the output buffer for sending to the host. It will be seen that there is a delay from CMD 224 to the slice prefetch shown in data 226. This delay results in an additional read delay of the first block of slice 1 being requested because the system must wait until all slices 0 are prefetched before accessing any content in slice 1.

Data 228 represents operations at the output driver between the memory device and the host. Data 226 shows one unified operation for reading a slice of data, while data 228 shows each block of data that is sent back to the host. Data 228 illustrates a configuration in which all data is sent back to the host in slices. Thus, even if the first data block of slice 1 is requested before the rest of the data in slice 0, all of slice 0 is returned to the host before all of the data for slice 1 is returned. Data is not returned until a request for all data for a slice has been received, thus maximizing bandwidth at the expense of latency. If slice 1 (data 226) is ready before all of slice 0 is sent to the host (data 228), then the response for slice 1 will be sent before slice 0.

FIG. 2C is a timing diagram of an example of a command sequence that focuses on the latency of the queue depth for a1 read with prefetching after a particular read is serviced, which may improve bandwidth to some extent for larger queue depths. Diagram 230 represents commands sent from the host and responded to from the memory device. The graph 230 represents commands from the host and operations at the media controller and media based on bandwidth optimization at the media, but allows some read latency optimization at the connection between the memory device and the host.

More specifically, the media controller reads a data block requested by the host, followed by a pre-fetch operation to read the remainder of the slice associated with the data block. This prefetching approach has similar effects of reducing address conflicts and improving bandwidth. According to fig. 230, once a sector is read, the media controller can respond to the host without having to wait until the entire slice is read (QD1 latency optimization).

CMD232 represents a command or read request sent by the host. Each element shown represents a separate command for a separate data block. These commands are indicated with shading to indicate what portion of the data the block is associated with. The legends indicate slice 0 of the shaded (a) block, slice 1 of the shaded (b) block, slice 2 of the shaded (c) block, and slice 3 of the shaded (d) block. The diagram 230 represents a scenario in which there are commands sent for individual blocks or sectors of different portions or slices that are interleaved with each other.

For example, CMD232 represents a request for a block of data from slice 0, followed by a request for data from slice 1. After a period of time, the host sends three consecutive requests for data blocks from slice 0. After a while, the host sends three consecutive requests for data blocks from slice 1, followed by a request for data from slice 2. The host then sends a command for the block of slice 2, followed by a command for the block of slice 3.

CMD 234 represents the command encoding from the controller to the storage medium. The host generates the command shown in command CMD232 to request a particular block of data. CMD 234 represents a command generated by the media controller for accessing the media. In one example, the media controller generates a command for the data block in response to a host request for the data block. The media controller then issues a pre-fetch command (labeled "P") to cause the media to read the rest of the data slice.

Data 236 represents operations provided to the buffer at the media. According to diagram 230, the read data may be provided to an output buffer for the requested data and the prefetched data may be placed in a prefetch cache.

Data 238 represents operations at an output driver between the memory device and the host. Data 238 shows that the first requested data block can be returned immediately after the read, which improves read latency. When other commands are received for the same slice, data may be returned from the prefetch cache.

Fig. 2D is a timing diagram of an example of a command sequence to switch between latency mode and bandwidth mode based on the number of pending commands. Diagram 240 represents commands sent from the host and responded to from the memory device. Diagram 240 represents commands from a host and operations at a media controller and media based on one read mode for latency optimization and one read mode for bandwidth optimization. The media controller dynamically determines how to switch between the two read modes. Diagram 240 represents a command exchange that may occur in accordance with the system 100 of fig. 1.

CMD 242 represents a command or read request sent by the host. Each element shown represents a separate command for a separate data block. These commands are indicated with shading to indicate what portion of the data the block is associated with. The legends indicate slice 0 of the shaded (a) block, slice 1 of the shaded (b) block, slice 2 of the shaded (c) block, and slice 3 of the shaded (d) block. The diagram 230 represents a scenario in which there are commands sent for individual blocks or sectors of different portions or slices that are interleaved with each other.

For example, CMD 242 represents a request for a block of data from slice 0, followed by a request for data from slice 1. After a period of time, the host sends three consecutive requests for data blocks from slice 0. After a while, the host sends three consecutive requests for data blocks from slice 1, followed by a request for data from slice 2. The host then sends a command for the block of slice 2, followed by a command for the block of slice 3.

CMD 244 represents command encoding from the controller to the storage medium. The host generates the command shown in CMD 242 to request a particular block of data. CMD 244 represents a command generated by the media controller for accessing the media. In one example, the media controller generates a command for the data block in response to a host request for the data block. The host may continue to generate commands for other data blocks of different slices until a command for a second data block of the same slice is received. Although the second block is used as a trigger for purposes of the example in fig. 240, it may be the third block, the fourth block, or some other nth request. In one example, graph 240 shows a configuration in which the queue depth is 2, which triggers a switch from delay mode to bandwidth mode. In one example, the queue depth may refer to the number of outstanding reads for the same slice of data.

The number of commands set as a threshold to trigger switching in read mode will depend on the system configuration, block and slice sizes relative to each other, or other factors. When the data block is 1/4, the total slice size, prefetching in response to the second command may be significant. If the data block is a slice size of 1/8 or 1/16 or some other size, the system may benefit by having a queue depth greater than 1 as a trigger.

As shown, in response to a request for the first block of slice 0, the media controller issues a command to read the requested block. The same is true for the first block of slice 1. In response to a request for the second block of slice 0, it will be observed that the media controller switches to issuing a slice read command in response to a sector read command. Thus, when the controller is in bandwidth mode, even a command for a single block will cause a slice to be read unless the slice has been prefetched. The slice read command instructs to switch from a read mode that focuses on latency to a read mode that focuses on bandwidth. Thus, in response to a request for the first block of slices 2 and 3, respectively, the media controller issues a command to prefetch the entire slice.

Data 246 represents operations at the medium for providing to the buffer. In the read latency mode, the buffer may be directly connected to the output buffer. In the bandwidth mode, the buffer may be a prefetch buffer to be sent to an output buffer for sending to the host. It will be seen that the first two commands are for the first two blocks, and the remaining commands are executed with prefetch reads.

Data 248 represents operations at the output driver between the memory device and the host. Data 248 shows that data accessed in the latency mode can be returned immediately after reading, which improves read latency. After switching to the bandwidth mode, the memory device may send data to the host in response to a request for data based on the prefetched data.

From diagram 240, it will be appreciated that the system can request a pattern from the host where a small low latency read or a high bandwidth sequential read is desired, and dynamically modify the memory access pattern to the media to improve performance for a particular workload. In one example, optimization occurs in two parts: the first part detects incoming workloads and the second part implements read caching with prefetchers, optimizing data access patterns to the media for sequential bandwidth without wasting bandwidth with unnecessary reads.

Dynamically changing read modes allows the system to dynamically optimize media access for different read queue depths and access modes. By eliminating media address conflicts, optimized switching may improve efficiency and speed of sequential reads from the media while still retaining latency optimization for small queue depth random reads (where the user is most likely to notice the latency impact).

In bandwidth mode, any host read or even a read of a portion of a slice causes the entire slice to be read and cached internally. An internal cache may refer to a cache on or controlled by the controller, or accessible to the controller. Future host reads of the pre-fetched slice may be completed from the internally cached data without generating another media read. In one example, once all reads are processed and the controller goes idle, as part of an idle process, or as part of a process of waking up from an idle state, the controller may automatically switch back from the bandwidth mode to the latency mode. In response to a threshold number of read commands pending in the command queue, the controller will again switch back to bandwidth mode. In one example, for a system with a queue that buffers both read and write commands, only read commands count a threshold and write commands in the queue may be ignored for consideration of whether to switch read modes. Similarly, the controller entering idle may refer to the read process entering idle and may not refer to whether there are more pending write commands.

FIG. 3 is a swim lane flow diagram of an example of switching between a latency mode and a bandwidth mode. The diagram is divided into three parts: fig. 3-1, 3-2, and 3-3 to accommodate page sizes. Flow 300 shows an example of a sequence diagram of diagram 240 of fig. 2D.

Beginning with fig. 3-1, at initialization point 302, access to the medium by the controller is based on a latency mode. At event 304, the host generates a command read a0, which is a request for sector 0 of slice 0. The controller receives the command and generates a read a0 command for the media. At event 306, the host generates a command read b0, which is a request for sector 0 of slice 1. The controller receives the command and generates a read b0 command for the media.

In one example, in response to the second read command, the controller triggers the bandwidth mode or bandwidth read mode at point 308. At event 310, the host generates a command read a1 for sector 1 of slice 0. For example, the host may actually generate this command before the controller enters bandwidth mode. However, to illustrate that the command will be processed in bandwidth mode, the command is shown after point 308 in flow 300. It will be appreciated that the command may actually be sent before switching read mode, but since this is done after a threshold number of reads, it will be processed in bandwidth mode.

Once in bandwidth mode, the controller triggers a prefetch of the entire slice in response to a read of a slice or a portion of a slice as long as data for the slice has not been prefetched. Multiple reads of the same slice may be queued and wait until the slice read is complete. In one example, once a slice read is performed and data in the internal prefetch buffer is available, data may be returned to the host for all queued reads for that slice. In one example, all subsequent reads of the prefetch slice will be completed from the internal prefetch buffer and no read from the medium will be initiated.

As shown, in response to read a1, the controller generates commands for media reads a1, a2, a3 to complete the pre-fetch for slice 0. At event 312 and event 314, the host generates read a2 and read a3, respectively. The cessation of the arrow at the controller in flow 300 instructs the controller to queue the command without sending another command to the media. Once the media has completed reading a0, at event 316, the media returns data a0 to the controller, which returns data to the host to trigger the completion of reading a 0.

Since the system is already in bandwidth mode, when the host generates the command read b1 at event 320, the controller will generate one or more commands to the media to read b1, b2, b3 to complete the pre-fetch of slice 1. At event 322, the media completes reading data b0 and sends it to the controller. The controller in turn provides data to the host to trigger read b0 to complete.

In one example, the host then generates commands read b2 and read b3 at event 324 and event 326, respectively. In response to these commands, the controller again queues the commands without accessing the medium, since the data slice has already been requested for prefetching.

Continuing with flow 300 at fig. 3-2, the medium completes the pre-fetch of the rest of slice 0 at event 328, returning data a1, a2, a 3. Since commands for all of these sectors have been received, the controller can buffer the data and send it to the host in order to trigger read a1 complete, read a2 complete, and read a3 complete. For any data sectors that have not been requested, the controller may continue to cache data until such time as a request for it may be received.

At event 330, the host generates a command read c 0. The controller may still be in bandwidth mode since there are still pending reads of slice 1. Thus, in response to reading c0, the controller may generate one or more commands for reading c0, c1, c2, c3 to read the entirety of slice 2 in response to the command for c 0. The flow 300 shows the subsequent command read c1 at event 332, read c2 at event 334, and read c3 at event 336. In response to these commands, the controller may queue the commands without accessing the medium.

At event 338, the media completes the pre-fetch of slice 1 data, sending it to the controller as data b0, b1, b 2. The controller may then respond to the request from the host by sending data to trigger read b1 complete, read b2 complete, and read b3 complete.

At event 340, the host generates a command read d0 for the data sector from slice 3. The system is still in bandwidth mode and thus the controller can generate one or more commands to the media for reading d0, d1, d2, d 3. At event 342, the media returns the data for slice 2, c0, c1, c2, c3, to the controller. Since all data has been requested, the controller can send the data to the host to trigger read c0 complete, read c1 complete, read c2 complete, and read c3 complete.

Continuing with flow 300 at FIG. 3-3, the host generates a command read d1 at event 344 and a read d2 at event 346. In response to these requests, the controller may simply queue the commands since the data for slice 3 has already been requested from the media. At event 348, the media returns data d0, d1, d2, d3 in response to the prefetch request. In one example, the return of data for slice 3 completes all pending read commands, and the controller may switch to a delayed mode at 350.

In the case of event 348, the controller has all data prefetches for slice 3. Consider that after data d0 is sent to the host to trigger the read d0 to complete, the host generates the command read d3 at event 352. The controller may continue to send data d1 and data d2 to trigger read d1 and read d2 completions, respectively, at the host. Also, since data d3 has been prefetched, the controller may simply return data at event 354 to trigger read d3 to complete.

It will be appreciated that since the controller is switched back to the latency mode, the controller is ready for a subsequent read burst and the initial latency mode may provide a good initial read latency. The delay can be improved by prefetching data once the burst is already in process and then switching to bandwidth mode.

In one example, the controller marks the data in the prefetch buffer as invalid when a write to the corresponding media address is received. In one example, when the prefetch buffer is full, the controller will evict prefetched data according to a cache eviction process. A simple cache eviction routine is a FIFO (first in first out) method, where the oldest entry will be deleted to prefetch data to process the new read.

FIG. 4 is a block diagram of an example of a system having a memory subsystem with near memory and far memory, and with an integrated near memory controller and an integrated far memory controller. System 400 provides one example of a system in accordance with system 100 of fig. 1, where processor 410 represents a host and remote memory 450 represents a memory device.

System 400 represents components of a multi-level memory system. System 400 specifically illustrates an integrated memory controller and an integrated remote memory controller. The integrated controller is integrated on the processor die or in the processor SOC package, or both.

Processor 410 represents an example of a processor die or processor SOC package. Processor 410 includes a processing unit 412, which may include one or more cores 420 to perform execution of instructions. In one example, core 420 includes a processor-side cache 422, which will include cache control circuitry and cache data storage. Cache 422 may represent any type of processor-side cache. In one example, individual cores 420 include local cache resources 422 that are not shared with other cores. In one example, multiple cores 420 share cache resources 422. In one example, individual cores 420 include local cache resources 422 that are not shared, and multiple cores 420 include shared cache resources. It should be understood that in the system shown, processor-side cache 422 may store both data and metadata on-die.

In one example, processor 410 includes a system fabric 430 to interconnect components of the processor system. System fabric 430 may be or include an interconnection between processing components 412, peripheral controls 436, one or more memory controllers (e.g., Integrated Memory Controller (iMC)432 and far memory controller 434), I/O controls (not specifically shown), a graphics subsystem (not specifically shown), or other components. The system fabric 430 enables the exchange of data signals between components. While system fabric 430 is generally shown as connected components, it should be understood that system 400 does not necessarily illustrate all of the components interconnected. The system fabric 430 may represent one or more mesh connections, central switching mechanisms, ring connections, a hierarchy of fabrics, or other topologies.

In one example, processor 410 includes one or more peripheral controllers 436 to disconnect resources from peripheral components or devices. In one example, peripheral control 436 represents a hardware interface to platform controller 460, which includes one or more components or circuits to control interconnections in a hardware platform or motherboard of system 400 to interconnect peripheral devices to processor 410. Component 462 represents any type of chip or interface or hardware element coupled to processor 410 via platform controller 460.

In one example, processor 410 includes an iMC 432 that embodies control logic coupled to near memory 440. In one example, near memory 440 is conventionally considered the main memory of system 400. Main memory refers to the memory resources that are accessed when a cache miss occurs on the last level of cache 422. The iMC 432 may include hardware circuitry and software/firmware control logic. In one example, near memory 440 represents a volatile memory resource.

In one example, processor 410 includes far memory controller 434, which represents control logic for controlling access to far memory 450. Far memory 450 represents a memory resource having a longer access time than that of near memory 440. In one example, far memory 450 includes non-volatile memory resources. Far memory controller 434 may include hardware circuitry and software/firmware control logic. Both the iMC 432 and the remote memory controller 434 may include scheduling logic to manage access to their respective memory resources.

Far memory 450 includes media 454, which represents storage media in which far memory 450 stores data for system 400. In one example, far memory 450 includes a controller 452 that represents a controller in far memory 450 that can dynamically determine whether to access media 454 with a first or second read access mode. One of the read access modes may be a latency-focused mode in which the controller 452 accesses the medium 454 to read data in response to a command from the processor 410 having the shortest read latency. In a latency-focused mode, the controller 452 accesses the medium 454 for smaller memory segments based on what is requested in a command from the processor 410. Another mode of access may be a bandwidth-focused mode, in which the controller 452 accesses the medium 454 to read data in larger segments, even when the processor 410 issues a command requesting a smaller segment. In the bandwidth-focused mode, the processor 452 prefetches data in response to a read command from the processor 410.

In one example, near memory 440 includes one or more DRAM memory modules as main memory. In one example, far memory 450 includes 3DXP memory. Thus, medium 454 may be or include a 3DXP memory, which is understood to have a slower but similar read time as compared to DRAM, and a significantly slower write time as compared to DRAM. However, 3DXP is non-volatile and therefore does not need to be refreshed like DRAM, allowing lower standby power consumption. The memory subsystems according to system 400 may include 3DXP far memory 450 and DRAM-based near memory 440. The total power usage will be improved and the access performance should be similar.

Instead of 3DXP, other memory technologies may be used, such as Phase Change Memory (PCM) or other non-volatile memory technologies. Non-limiting examples of non-volatile memory may include any one or combination of the following: solid state memory (e.g., planar or 3D NAND flash memory or NOR flash memory), memory devices using chalcogenide phase change materials (e.g., chalcogenide glass), byte addressable non-volatile memory devices, ferroelectric memory, silicon-oxygen-nitrogen-oxygen-silicon (SONOS) memory, polymer memory (e.g., ferroelectric polymer memory), ferroelectric transistor random access memory (Fe-TRAM), ovonic memory, nanowire memory, electrically erasable programmable read-only memory (EEPROM), other various types of non-volatile Random Access Memory (RAM), and magnetic storage memory. In some examples, the 3D cross-point memory may include a transistor-less stackable cross-point architecture, wherein the memory cells are located at intersections of word lines and bit lines and are individually addressable, and wherein the bit storage is based on a change in bulk resistance.

FIG. 5 is a block diagram of an example of a non-volatile storage system having access mode control to change control based on a number of queued read requests. System 500 provides one example of a system in accordance with system 100 of fig. 1, where host 510 represents a host and NVM device 520 represents a memory device.

The system 500 includes an NVM (non-volatile memory) device 520 coupled with a host 510. In one example, NVM device 520 represents an nth level memory or byte addressable nonvolatile storage device coupled to a main memory bus. In one example, the NVM device 520 is a Solid State Drive (SSD). Host 510 represents the host hardware platform connected to NVM device 520. Host 510 includes a CPU (central processing unit) or other processor as a host processor for executing host OS 512. The host 510 may include a motherboard or hardware board, chipset, or other hardware component to interconnect the host 510 to the NVM device 520.

System 500 shows the logical layers of a host and NVM device. Host OS 512 represents a host operating system or software platform for a host. Host OS 542 may include a platform on which applications, services, agents, and/or other software execute and are executed by processors. The file system 514 represents control logic for controlling access to the NVM device 520. The file system 514 may manage what addresses or memory locations are used to store what data. There are many known file systems, and the file system 514 may implement a known file system or other proprietary system. In one example, file system 514 is part of host OS 542.

Storage drives 516 represent one or more system level modules that control the hardware of host 510. In one example, the driver 516 includes a software application to control the interface to the NVM device 520 and, thus, the hardware of the NVM device 520. The storage drive 516 may provide a communication interface between the host and the NVM device.

NVM device 520 represents a storage drive that includes a primary non-volatile (NV) medium (represented by NV medium 524) to store data. Volatile media 522 represents smaller and faster media to act as a buffer or cache for NV media 524. The NVM device 520 includes a controller 530 to control access to the buffer 522 and NV media 524. Controller 530 represents the hardware and control logic within NVM device 520 to perform control of the media.

Controller 530 includes firmware 534, which represents control software/firmware for the controller. In one example, the controller 530 includes a host interface 532, which represents an interface to the host 510. In one example, the controller 530 includes an NV interface 536, which represents an interface to the volatile media 522 and NV media 524. It will be appreciated that the NV interface 536 may interface with a volatile memory device as a buffer for NV media.

Interfaces 532 and 536 include controls that execute on the hardware of controller 530. It will be understood that controller 530 includes hardware for interfacing with host 510, which may be considered to be controlled by host interface software/firmware 532. Also, it should be understood that the controller 530 includes hardware for interfacing with the volatile media 522 and the NV media 534. In one example, code for host interface 532 may be part of firmware 534. In one example, the code for NV interface 536 may be part of firmware 534.

In one example, controller 530 includes error control 538 to handle data errors in accessed data and special cases (corner cases) in terms of adherence to signaling and communication interfaces. In one example, error control 538 is implemented in hardware. In one example, error control 538 is implemented within firmware 534. In one example, error control 538 is implemented as a combination of hardware and firmware.

According to any of the examples described herein, the access mode control 540 represents logic within the controller 530 to dynamically change between a read mode that focuses on read latency and a read mode that focuses on bandwidth. With the focus on latency, the controller 530 only accesses data requested from the host 510 as soon as the data can be accessed and returned. With bandwidth focused on, the controller accesses more data than requested unless the requested data has already been prefetched, and buffers or buffers the data to improve the output bandwidth back to the host 510. In one example, controller 530 includes parameters for determining when to trigger a handoff from delay-focused to bandwidth-focused.

In one example, NV medium 524 comprises a cross-point memory medium. In one example, the NV medium 524 includes a stacked memory device having multiple layers of storage units. In one example, the NVM device 520 includes a PCB (printed circuit board) or module on which components of the device are mounted. In one example, controller 530 represents a media controller on a PCB module that includes a plurality of individual persistent media dies. The one or more persistent media die may include a multi-layer stack or a 3D memory stack.

References to a memory device refer to memory whose state (and thus the data stored thereon) is indeterminate when the device is powered down. Non-volatile memory refers to memory whose state is determinate even if the device is powered off. Dynamic volatile memories require refreshing of data stored in the device to maintain state. Volatile memory as referred to herein may include DRAM (dynamic random access memory) devices, or some variation such as synchronous DRAM (sdram). The memory subsystem may be compatible with a number of memory technologies, such as DDR4 (double data rate (DDR) version 4, JESD79, initial specifications published by JEDEC in month 9 2012), LPDDR4 (low power DDR version 4, JESD209-4, originally published by JEDEC in month 8 2014), WIO 2(wide I/O2 (WideIO2), JESD229-2, originally published by JEDEC in month 8 2014), HBM (high bandwidth memory DRAM, JESD235A, originally published by dec in month 11 2015), DDR5(DDR version 5, currently discussed by JEDEC), LPDDR version 5(LPDDR version 5, JESD209-5, originally published by JEDEC in month 2 2019), HBM2((HBM version 2), currently discussed by JEDEC), or other memory technologies or combinations of memory technologies, as well as technologies based on such derived specifications or extensions.

References to non-volatile memory devices or persistent memory may include non-volatile memory devices that are block addressable memory devices, such as NAND or NOR technology. Further, non-volatile memory may refer to byte-addressable memory, such as three-dimensional cross-point memory devices, other byte-addressable non-volatile memory devices, or memory devices that use chalcogenide phase change material (e.g., chalcogenide glass) or memory devices that store data based on the resistance state of a storage medium. In one example, the memory device may be or include multi-threshold level NAND flash memory, NOR flash memory, single or multi-level Phase Change Memory (PCM) or phase change memory with Switches (PCMs), resistive memory, nanowire memory, ferroelectric transistor random access memory (FeTRAM), Magnetoresistive Random Access Memory (MRAM) memory incorporating memristor technology, or Spin Transfer Torque (STT) -MRAM, or a combination of any of the above, or other memory.

FIG. 6 is a block diagram of an example of a computing system in which switching between read modes based on queue depth may be implemented. System 600 represents a computing device according to any example herein, and can be a laptop computer, desktop computer, tablet computer, server, gaming or entertainment control system, embedded computing device, or other electronic device. System 600 provides an example of a system according to system 100.

More specifically, processor 610 and the host OS executed by the processor may represent a host, with memory resources in memory subsystem 620 or memory resources in storage subsystem 680 as memory devices. In one example, according to any example herein, system 600 includes an access mode control 690 that represents a component that enables dynamic switching between different read modes of a memory medium. In one example, the access mode control 690 may be part of the controller 682 of the storage subsystem 680. In one example, the access mode control 690 may be part of a controller on a memory device of the memory 630, where the controller is not specifically shown. The controller will be understood to be a media controller, which may be different from the memory controller 622. In one example, the memory controller may comprise a media controller.

System 600 includes a processor 610, which may include any type of microprocessor, Central Processing Unit (CPU), Graphics Processing Unit (GPU), processing core, or other processing hardware or combination to provide instruction processing or execution for system 600. The processor 610 controls the overall operation of the system 600 and may be or include one or more programmable general purpose or special purpose microprocessors, Digital Signal Processors (DSPs), programmable controllers, Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), or a combination of such devices.

In one example, the system 600 includes an interface 612 coupled to the processor 610, which may represent a higher speed interface or a high throughput interface for system components (e.g., the memory subsystem 620 or the graphics interface component 640) that require higher bandwidth connections. Interface 612 represents interface circuitry, which may be a separate component or integrated onto the processor die. The interface 612 may be integrated as a circuit on the processor die or as a component on a system on a chip. The graphical interface 640 interfaces with graphical components, if present, to provide a visual display to a user of the system 600. Graphics interface 640 may be a separate component or integrated onto a processor die or system on a chip. In one example, the graphics interface 640 may drive a High Definition (HD) display that provides output to a user. In one example, the display may comprise a touch screen display. In one example, the graphics interface 640 generates a display based on data stored in the memory 630 or based on operations performed by the processor 610, or both.

Memory subsystem 620 represents the main memory of system 600 and provides storage for code to be executed by processor 610 or data values to be used in executing routines. Memory subsystem 620 may include one or more memory devices 630, such as Read Only Memory (ROM), flash memory, one or more variations of Random Access Memory (RAM), such as DRAM, or other memory devices, or a combination of these devices. Memory 630 stores and hosts, among other things, an Operating System (OS)632 to provide a software platform for executing instructions in system 600. Additionally, applications 634 may execute from memory 630 on the software platform of the OS 632. Applications 634 represent programs that have their own operating logic to perform one or more functions. Process 636 represents an agent or routine that provides ancillary functionality to OS 632 or one or more applications 634 or a combination. The OS 632, applications 634, and processes 636 provide software logic to provide functionality for the system 600. In one example, memory subsystem 620 includes memory controller 622, which is a memory controller used to generate and issue commands to memory 630. It will be appreciated that the memory controller 622 may be a physical part of the processor 610 or a physical part of the interface 612. For example, memory controller 622 may be an integrated memory controller that is integrated with processor 610 on a circuit, such as on a processor die or system on a chip.

Although not specifically shown, it will be understood that system 600 may include one or more buses or bus systems between devices, such as a memory bus, a graphics bus, an interface bus, or others. A bus or other signal line may communicatively or electrically couple the components together or both. A bus may include physical communication lines, point-to-point connections, bridges, adapters, controllers, or other circuits or combinations. The bus may include, for example, one or more of a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), other buses, or a combination thereof.

In one example, system 600 includes an interface 614 that can be coupled to interface 612. Interface 614 may be a slower speed interface than interface 612. In one example, interface 614 represents interface circuitry, which may include separate components and integrated circuits. In one example, a plurality of user interface components or peripheral components or both are coupled to the interface 614. Network interface 650 provides system 600 with the ability to communicate with remote devices (e.g., servers or other computing devices) over one or more networks. Network interface 650 may include an ethernet adapter, a wireless interconnect component, a cellular network interconnect component, USB (universal serial bus), or other wired or wireless standard-based or proprietary interface. Network interface 650 may exchange data with a remote device, which may include transmitting data stored in memory or receiving data to be stored in memory.

In one example, system 600 includes one or more input/output (I/O) interfaces 660. The I/O interface 660 can include one or more interface components through which a user interacts with the system 600 (e.g., audio, alphanumeric, tactile/touch, or other engagement). Peripheral interface 670 may include any hardware interface not specifically mentioned above. Peripheral devices generally refer to devices that are independently connected to the system 600. A slave connection is a connection in which the system 600 provides a software platform or a hardware platform or both on which to perform operations and interact with a user.

In one example, system 600 includes a storage subsystem 680 for storing data in a nonvolatile manner. In one example, in some system implementations, at least some components of storage 680 may overlap with components of memory subsystem 620. The storage subsystem 680 includes a storage device 684, which may be or include any conventional medium for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid state, or optical based disks or combinations. The storage 684 holds code or instructions and data 686 in a persistent state (i.e., values are retained despite system 600 powering down). Although the memory 630 is typically an execution or manipulation memory that provides instructions to the processor 610, the storage 684 may generally be considered to be a "memory". Although the storage 684 is non-volatile, the memory 630 can comprise volatile memory (i.e., the value or state of data is indeterminate if the system 600 is powered down). In one example, the storage subsystem 680 includes a controller 682 for interfacing with a storage device 684. In one example, controller 682 is interface 614 or a physical part of processor 610, or may include circuitry or logic in both processor 610 and interface 614.

The power supply 602 provides power to the components of the system 600. More specifically, the power supply 602 generally interfaces with one or more power supplies 604 in the system 600 to provide power to the components of the system 600. In one example, the power supply 604 includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. Such an AC power source may be a renewable energy (e.g., solar) power source 602. In one example, the power supply 602 includes a DC power supply, such as an external AC to DC converter. In one example, the power source 602 or power supply 604 includes wireless charging hardware to charge via proximity to a charging field. In one example, the power source 602 may include an internal battery or fuel cell source.

Fig. 7 is a block diagram of an example of a mobile device in which switching between read modes based on queue depth may be implemented. System 700 represents a mobile computing device, such as a computing tablet, mobile or smart phone, wearable computing device, or other mobile or embedded computing device. It will be understood that certain components are shown generally, and not all components of such a device are shown in the system 700. System 700 provides an example of a system according to system 100.

More specifically, processor 710 and the host OS executed by the processor may represent a host, with memory resources in memory subsystem 760 as memory devices. In one example, according to any example herein, system 700 includes an access mode control 790 in memory subsystem 760 that represents a component for enabling dynamic switching between different read modes of a storage medium. In one example, access mode control 790 may be part of a controller of NV memory 766. In one example, memory subsystem 760 includes NV memory 766, which represents memory having non-volatile media that can persistently store data. In one example, the media controller for NV memory 766 includes an access mode control 790. The controller used to access the media and implement the access mode control is understood to be a media controller, which may be different from the memory controller 762. In one example, the memory controller may comprise a media controller.

The system 700 includes a processor 710 that performs the primary processing operations of the system 700. Processor 710 may include one or more physical devices such as microprocessors, application processors, microcontrollers, programmable logic devices, or other processing units. The processing operations performed by processor 710 include the execution of an operating platform or operating system on which applications and device functions are executed. The processing operations include operations related to I/O (input/output) of a human user or other device, operations related to power management, operations related to connecting the system 700 to another device, or a combination. Processing operations may also include operations related to audio I/O, display I/O, or other interfacing or combining operations. The processor 710 may execute data stored in the memory. The processor 710 may write or edit data stored in the memory.

In one example, the system 700 includes one or more sensors 712. Sensor 712 represents an embedded sensor or an interface or combination to an external sensor. The sensors 712 enable the system 700 to monitor or detect one or more conditions of the environment or device in which the system 700 is implemented. The sensors 712 may include environmental sensors (e.g., temperature sensors, motion detectors, light detectors, cameras, chemical sensors (e.g., carbon monoxide, carbon dioxide, or other chemical sensors), pressure sensors, accelerometers, gyroscopes, medical or physiological sensors (e.g., a biosensor, a heart rate monitor, or other sensor for detecting a physiological property), or other sensors, or a combination thereof the sensors 712 may also include sensors for a biometric identification system, such as a fingerprint recognition system, a facial detection or recognition system, or other system that detects or recognizes a characteristic of a user. One or more sensors 712 are coupled to the processor 710 via another component of the system 700.

In one example, system 700 includes an audio subsystem 720, which represents hardware (e.g., audio hardware and audio circuitry) and software (e.g., drivers, codecs) components associated with providing audio functionality to a computing device. The audio functions may include speaker or headphone output, and microphone input. Devices for such functions may be integrated into system 700 or connected to system 700. In one example, a user interacts with the system 700 by providing audio commands that are received and processed by the processor 710.

Display subsystem 730 represents hardware (e.g., display devices) and software (e.g., drivers) components that provide visual displays for presentation to a user. In one example, the display includes a haptic assembly or touch screen element for user interaction with the computing device. Display subsystem 730 includes a display interface 732, which includes a particular screen or hardware device for providing a display to a user. In one example, the display interface 732 includes logic separate from the processor 710 (e.g., a graphics processor) to perform at least some processing related to the display. In one example, display subsystem 730 includes a touch screen device that provides both output and input to a user. In one example, display subsystem 730 includes a High Definition (HD) or Ultra High Definition (UHD) display that provides output to a user. In one example, the display subsystem includes or drives a touch screen display. In one example, display subsystem 730 generates display information based on data stored in memory or based on operations performed by processor 710, or both.

I/O controller 740 represents the hardware devices and software components related to interaction with a user. I/O controller 740 may operate to manage hardware that is part of audio subsystem 720 or display subsystem 730, or both. In addition, I/O controller 740 illustrates a connection point for additional devices connected to system 700 through which a user may interact with the system. For example, devices that may be attached to system 700 may include a microphone device, a speaker or stereo system, a video system or other display device, a keyboard or keypad device, or other I/O devices (e.g., card readers or other devices) for use with a particular application.

As described above, I/O controller 740 may interact with audio subsystem 720 or display subsystem 730, or both. For example, input through a microphone or other audio device may provide input or commands for one or more applications or functions of the system 700. In addition, audio output may be provided instead of or in addition to display output. In another example, if the display subsystem includes a touch screen, the display device also acts as an input device, which may be managed, at least in part, by I/O controller 740. Additional buttons or switches may also be present on system 700 to provide I/O functions managed by I/O controller 740.

In one example, I/O controller 740 manages devices such as accelerometers, cameras, light or other environmental sensors, gyroscopes, Global Positioning Systems (GPS) or other hardware that may be included in system 700 or sensors 712. The input may be part of direct user interaction, or environmental input may be provided to the system to affect its operation (e.g., filter noise, adjust display for brightness detection, apply flash for camera, or other features).

In one example, system 700 includes power management 750 that manages battery power usage, battery charging, and functions related to power saving operations. Power management 750 manages power from a power supply 752 that provides power to the components of system 700. In one example, the power supply 752 includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. Such AC power may be a renewable energy source (e.g., solar, motion-based electricity). In one example, the power supply 752 includes only DC power, which may be provided by a DC power supply such as an external AC-DC shifter. In one example, the power supply 752 includes wireless charging hardware to charge via proximity to a charging field. In one example, the power source 752 may include an internal battery or fuel cell source.

Memory subsystem 760 includes a memory device 762 for storing information in system 700. Memory subsystem 760 can include non-volatile (state does not change if the memory device is powered down) or volatile (state is indeterminate if the memory device is powered down) memory devices or combinations. Memory 760 may store application data, user data, music, photos, documents, or other data, as well as system data (whether long-term or temporary) related to the execution of the applications and functions of system 700. In one example, memory subsystem 760 includes memory controller 764 (which may also be considered part of the control of system 700, and potentially as part of processor 710). Memory controller 764 includes a scheduler to generate and issue commands to control access to memory devices 762.

Connectivity 770 includes hardware devices (e.g., wireless or wired connectors and communication hardware, or a combination of wired and wireless hardware) and software components (e.g., drivers, protocol stacks) to enable the system 700 to communicate with external devices. The external devices may be separate devices (e.g., other computing devices, wireless access points, or base stations) as well as peripheral devices (e.g., headphones, printers, or other devices). In one example, the system 700 exchanges data with an external device for storage in memory or display on a display device. The exchanged data may include data to be stored in the memory or data already stored in the memory to read, write or edit the data.

Connectivity 770 may include a variety of different types of connectivity. In general, system 700 is shown with cellular connectivity 772 and wireless connectivity 774. Cellular connectivity 772 generally refers to cellular network connectivity provided by a wireless operator, such as via GSM (global system for mobile communications) or variants or derivations, CDMA (code division multiple access) or variants or derivations, TDM (time division multiplexing) or variants or derivations, LTE (long term evolution-also referred to as "4G") or other cellular service standards. Wireless connectivity 774 refers to wireless connectivity that is not cellular, and may include personal area networks (e.g., bluetooth), local area networks (e.g., WiFi), or wide area networks (e.g., WiMax), or other wireless communications, or a combination. Wireless communication refers to the transmission of data through a non-solid medium through the use of modulated electromagnetic radiation. Wired communication is performed through a solid communication medium.

Peripheral connection 780 includes hardware interfaces and connectors, as well as software components (e.g., drivers, protocol stacks) to make the peripheral connection. It will be understood that system 700 may be a peripheral device ("to" 782) to other computing devices, as well as having a peripheral device (from "784") connected thereto. The system 700 typically has a "docking" connector to connect to other computing devices for purposes such as managing (e.g., downloading, uploading, altering, synchronizing) content on the system 700. Additionally, a docking connector may allow the system 700 to connect to certain peripherals that allow the system 700 to control the output of content to, for example, an audiovisual or other system.

In addition to proprietary docking connectors or other proprietary connection hardware, the system 700 may also make peripheral connections 780 via generic or standards-based connectors. Common types may include Universal Serial Bus (USB) connectors (which may include any of a number of different hardware interfaces), DisplayPort including minidisplayport (mdp), High Definition Multimedia Interface (HDMI), or other types.

Generally, with respect to the description herein, in one example, a memory device includes: a persistent storage media accessible as a block of data, the block of data having a size at least equal to an access size of the volatile media cache; and a media controller for controlling access to the persistent storage media, the media controller for dynamically switching between a first read mode and a second read mode based on a number of pending read commands in a command queue, the first read mode for accessing only data blocks addressed in the pending commands and the second read mode for accessing data blocks addressed in the pending commands and pre-fetching consecutive data blocks of the data blocks.

In one example, the persistent storage medium includes a cross-point memory medium having multiple layers of memory cells. In one example, a media controller includes a controller on a Printed Circuit Board (PCB) module having a plurality of persistent media dies. In one example, the media controller is to preset to the first read mode and switch to the second read mode in response to a threshold number of read commands pending in the command queue. In one example, the storage device further comprises: a register to store a programmable value to set the threshold number for triggering switching from the first read mode to the second read mode. In one example, the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands. In one example, the memory device further includes a prefetch buffer to store the prefetched data block. In one example, in the second read mode, the media controller is to return a data block within the data block that has been prefetched from the prefetch buffer in response to a read command addressed to the data block. In one example, in the second read mode, in response to a read command addressed to a data block within the data block, the media controller is to queue the read command to wait for a prefetch of the data block.

Generally, with respect to the description herein, in one example, a system comprises: a volatile memory device including a volatile medium as a near memory for the processor; and a persistent memory device comprising a persistent storage medium as a far memory of the processor, the persistent memory device comprising: a persistent storage medium as a remote memory of the processor, the persistent storage medium accessible as a data block, the data block having a size at least equal to an access size of the volatile medium; and a media controller for controlling access to the persistent storage media, the media controller for dynamically switching between a first read mode and a second read mode based on a number of pending read commands in a command queue, the first read mode for accessing only data blocks addressed in the pending commands and the second read mode for accessing data blocks addressed in the pending commands and pre-fetching consecutive data blocks of the data blocks.

In one example, the media controller is to preset to the first read mode and switch to the second read mode in response to a threshold number of read commands pending in the command queue. In one example, the persistent memory further comprises: a register to store a programmable value to set the threshold number to trigger a switch from the first read mode to the second read mode, wherein the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands. In one example, the persistent memory includes a hardcoded value to set the threshold number for triggering a switch from the first read mode to the second read mode, wherein the media controller is to switch from the second read mode to the first read mode in response to the command queue having no pending read commands. In one example, the persistent memory further comprises: a prefetch buffer for storing prefetched data blocks. In one example, in the second read mode, in response to a read command addressed to a data block within the data block that has been prefetched, the media controller is to return the data block from the prefetch buffer; and wherein in the second read mode, in response to a read command addressed to a data block within the data block, the media controller is to queue the read command to await a prefetch of the data block. In one example, the system further comprises one or more of: a host processor device coupled to the volatile memory device and the persistent memory device; a display communicatively coupled to the host processor; a network interface communicatively coupled to the host processor; or a battery for powering the system.

Generally, with respect to the description herein, in one example, a method for storing data comprises: receiving a read command from a host controller; determining whether a number of pending read commands in a command queue meets or exceeds a threshold number of read commands in the command queue; and dynamically switching from a first read mode to a second read mode in response to determining that the number of read commands meets or exceeds the threshold number, the first read mode being for accessing only data blocks addressed in pending commands and the second read mode being for accessing data blocks addressed in pending commands and pre-fetching successive data blocks of a data block.

In one example, dynamically switching from the first read mode to the second read mode includes: presetting to the first reading mode; and switching to the second read mode in response to a threshold number of read commands pending in the command queue. In one example, the method further comprises: storing a programmable value in a register to set the threshold number for triggering a switch from the first read mode to the second read mode, wherein switching from the second read mode to the first read mode comprises switching in response to the command queue having no pending read commands. In one example, the method further comprises storing the prefetched data block in a prefetch buffer. In one example, in the second read mode, in response to a read command addressed to a data block within the data block that has been prefetched, returning the data block from the prefetch buffer; and, in response to a read command addressed to a data block within the data block, queuing the read command to await a prefetch of the data block.

The flow diagrams as shown herein provide examples of sequences of various process actions. The flow diagrams may indicate operations to be performed by software or firmware routines and physical operations. The flow diagrams may illustrate examples of implementations of states of a Finite State Machine (FSM), which may be implemented in hardware and/or software. Although shown in a particular order or sequence, the order of the operations may be modified unless otherwise specified. Thus, the illustrated figures should be understood only as examples, and the process may be performed in a different order, and some actions may be performed in parallel. Further, one or more acts may be omitted; thus, not all implementations will perform all operations.

To the extent various operations or functions are described herein, they may be described or defined as software code, instructions, configurations, and/or data. The content may be directly executable files ("target" or "executable" form), source code, or difference code ("delta" or "patch" code). The software content described herein may be provided via an article of manufacture on which the content is stored or by a method of operating a communication interface to transmit data via the communication interface. A machine-readable storage medium may cause a machine to perform the functions or operations described, and includes any mechanism for storing information in a form accessible by a machine (e.g., a computing device, an electronic system, etc.), such as recordable/non-recordable media (e.g., Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.). A communication interface includes any mechanism for interfacing with any of a hardwired, wireless, optical, etc. medium to communicate with another device, such as a memory bus interface, a processor bus interface, an internet connection, a disk controller, etc. The communication interface may be configured by providing configuration parameters and/or sending signals to prepare the communication interface to provide data signals describing the software content. The communication interface may be accessed by one or more commands or signals sent to the communication interface.

The various components described herein may be means for performing the described operations or functions. Each component described herein includes software, hardware, or a combination thereof. These components may be implemented as software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), etc.), embedded controllers, hardwired circuitry, and so forth.

Various modifications, in addition to those described herein, can be made to the disclosed and embodiments of the invention without departing from the scope of the invention. Accordingly, the illustrations and examples herein should be construed in an illustrative, and not a restrictive, sense. The scope of the invention should be measured solely by reference to the claims that follow.