湖北网站建设服务公司,新浪云怎么做自己的网站,自己怎么建设手机网站首页,网站上的3d产品展示怎么做分布式数据库系统
针对于海量数据#xff0c;可扩展#xff0c;高吞吐量#xff0c;低时延
不支持关系模型
通过row和column进行索引#xff0c;row和column可以是任意字符串
所存储的数据也是字符串 Bigtable是一个map#xff0c;value是array of bytes#xff0c;通…
分布式数据库系统
针对于海量数据可扩展高吞吐量低时延
不支持关系模型
通过row和column进行索引row和column可以是任意字符串
所存储的数据也是字符串 Bigtable是一个mapvalue是array of bytes通过row key, column key, timestamp检索。
(row:string, column:string, time:int64) -- string 读写操作对于行是原子性的
The row range for a table is dynamically partitioned.
Each row range is called a tablet, which is the unit of distribution and load balancing.
一个tablet包含多行 列族是访问控制的基本单位
同一列族的数据通常是相同类型的对同一列族的数据进行压缩
列族属于模式数量有限in the hundreds at most很少改变
列的数量是不限制的have an unbounded number of columns
列键 column key命名为family:qualifier 时间戳timestamp
可通过HBase系统指定也可在客户端指定
可存储最新n条数据也可存储最近几天的数据 基于GFS存储log和data files
SSTable文件格式来存储Bigtable data
Internaly, each SSTable contains a sequence of blocks(typically each block is 64KB in size, but this is configurable).
A block index (stored at the end of the SSTable) is used to locate blocks; the index is loaded into memory when the SSTable is opened. Chubby provides a namespace that consists of directories and small files.
Each directory or file can be used as a lock, and reads and writes to a file are atomic. Chubby的职责
1、to ensure that there is at most one active master at any time
2、to store the bootstrap location of Bigtable data
3、to discover tablet servers and finallize tablet server deaths
4、to store Bigtable schema information (the column family information for each table)
5、to store access control lists
如果Chubby持续一段时间不能访问Bigtable becomes unavialiable。 实现包含三个部分
a library that is linked into every client
one master server
many tablet servers (can be dynamically added or removed) master server的职责
1、assigning tablets to tablet servers
2、detecting the addition and expiration of tablet servers
3、balancing tablet-server load
4、garbage collection of files in GFS
5、handles schema changes such as table and column family creations teblet server的职责
1、manages a set of tablets (typically we have somewhere between ten to thousand tablets per tablet server)
2、handles read and write requests to the teblets that is loaded
3、splits tablets that have grown too large 每个table包含若干tablets每个tablet对应多行 通过3层类B树实现tablet的索引
第一层是保存在Chubby中的文件记录的是root tablet的位置
第二层为root tablet包含METADATA表的tablets信息为了保证三层结构root table只有一个不进行split。
第二层为METADATA包含的是user tablets的位置以tablets table identifier和its end row为keyvalue为对应tablet的位置 root tablet实际是第一个METADATA表的tablet。 Locality groups:
用户可以将同时访问的列族设置为一个locality group每个locality group作为一个SSTable存储 Compression
可以将包含locality group的SSTable文件进行压缩存储以节省空间 Caching for read performance:
Higher-level cache: the key-value pairs returned by the SSTable
Lower-level cache: SSTables blocks Bloom filters
A read operation has to read from all SSTables that make up the state of a tablet.
过滤器可以减少磁盘访问量
