我自己也有一套
先赞再说~
split 拆分有啥问题的吗？？
mark111111111
能具体说说么？
.split(’@’)
这种拆分 字符串 和
protobuffer binary.ts 哪种性能更优
自己定义 二维数组就够用了
看个人喜欢吧
json 传递 的话会多 { } " : , ] 几个字符
不只是这样，我来说说与json字符串的对比吧：
省略json中的结构
像你说的那样，{ } " : , ]，这些都没有了
省略key
比如{ age: 60000 }，这里用protobuffer或binary.ts转成二进制后，age是被丢弃的，只保留60000，因为早已定义过结构了
number类型
比如{ age: 60000 }转成字符串的话，光60000就占了5个字节，但是在二进制中它其实只用1个字节就能表示了
boolean类型
true false这种转成字符串的话也是占了很多位，在binary.ts中只占1位
string类型
这个区别不大
是 1 字节，哈哈哈
字符串得看看平台是否支持 TextDecoder，支持的话可以直接支持 utf-8。楼主这里只做了定长的解码，对包含中文的数据来说数据量会比 utf-8 大一些。当然要用 js 做 utf-8 解码也行，就是比较复杂。
抱歉楼主，论坛有设置，超过 60 天就无法编辑原帖了。
了解了二进制的原理，和语言本身没有关系的 后端用c++ 一样可以用
我昨天正好看到protobufjs有一段处理，让我 借鉴 过来了 ，想着更新一下，发现没法编辑了
【优化】将String8、String16、String32简化为String
【优化】将Object8、Object16、Object32简化为Object
【优化】将Array8、Array16、Array32简化为Array
【新增】新增一个Base64类型，用来以更少的字节数存储base64格式字符串
性能上在nodejs上和protobuffer还有不小的差距，有时间再优化一把
/**
* 类似于protobuffer，但此库及其精简且专为js打造
* @author zp
* @version 1.1.0
*/
/**
* [注] nodejs环境下通过Uint8Array将ArrayBuffer和Buffer互相转换
* @example
* // Buffer ---> ArrayBuffer
* function toArrayBuffer(buf) {
* var ab = new ArrayBuffer(buf.length);
* var view = new Uint8Array(ab);
* for (var i = 0; i < buf.length; ++i) {
* view[i] = buf[i];
* }
* return ab;
* }
* // ArrayBuffer ---> Buffer
* function toBuffer(ab) {
* var buf = new Buffer(ab.byteLength);
* var view = new Uint8Array(ab);
* for (var i = 0; i < buf.length; ++i) {
* buf[i] = view[i];
* }
* return buf;
* }
*/
/**
* @example
* var { registerProto, Type, encode, decode, singleArray } = binary;
* registerProto(1001, {
* name: Type.String,
* age: Type.Uint8,
* sex: Type.Uint8
* })
* registerProto(1002, {
* info: 1001,
* gold: Type.Uint16,
* items: [Type.Uint16, Type.String]
* })
* registerProto(1003, {
* array0: Type.Array,
* array1: singleArray(1002),
* array2: singleArray([1001, 1002]),
* array3: singleArray(Type.Uint16),
* array4: singleArray([Type.Uint16, Type.String])
* })
* var buffer = encode({ name: 'Mary', age: 18, sex: 0 }, 1001);
* decode(buffer);
* var buffer = encode({ info: { name: 'Mary', age: 18, sex: 0 }, gold: 10, array: [100, 2, 3] }, 1002);
* decode(buffer);
* var buffer = encode({
* array0: ['你好啊','我很好'],
* array1: [{ info: { name: 'James', age: 30, sex: 1 }, gold: 10, array: [100, 2, 3] }],
* array2: [[{}, { info: { name: 'Mary', age: 18, sex: 0 }, gold: 10, array: [100, 2, 3] }]],
* array3: [568],
* array4: [[0, '零'], [1, '一'], [2, '二'], [3, '三']]
* }, 1003);
* decode(buffer);
*/
/**
* https://segmentfault.com/a/1190000014533505 中提到如果服务器开启了压缩了话，需要进行解压操作，并推荐了pako.js
* (具体也不知道这个压缩怎么回事，测试的时候也没遇到这个问题，先写注释记下来)
* @see https://github.com/nodeca/pako/edit/master/dist/pako.js
* @example
* let compressdata = new Uint8Array(buffer, byteOff, length);
* let uncompress = pako.inflate(compressdata);//解压数据
* let uncompressdata = uncompress.buffer;// ArrayBuffer {}
* let dataViewData = new DataView(uncompressdata, 0);//解压后数据
*/
/**
* A minimal base64 implementation for number arrays.
* @memberof util
* @namespace
*/
class Base64 {
// Base64 encoding table
private b64 = new Array(64);
// Base64 decoding table
private s64 = new Array(123);
private invalidEncoding = "invalid encoding";
constructor() {
// 65..90, 97..122, 48..57, 43, 47
for (var i = 0; i < 64;) {
this.s64[this.b64[i] = i < 26 ? i + 65 : i < 52 ? i + 71 : i < 62 ? i - 4 : i - 59 | 43] = i++;
}
}
/**
* Calculates the byte length of a base64 encoded string.
* @param {string} string Base64 encoded string
* @returns {number} Byte length
*/
length(string: string): number {
var p = string.length;
if (!p)
return 0;
var n = 0;
while (--p % 4 > 1 && string.charAt(p) === "=")
++n;
return Math.ceil(string.length * 3) / 4 - n;
};
/**
* Encodes a buffer to a base64 encoded string.
* @param {DataView} buffer Source buffer
* @param {number} start Source start
* @param {number} end Source end
* @returns {string} Base64 encoded string
*/
read(buffer: DataView, start: number, end: number): string {
var parts = null,
chunk = [];
var i = 0, // output index
j = 0, // goto index
t; // temporary
while (start < end) {
var b = buffer.getUint8(start++);
switch (j) {
case 0:
chunk[i++] = this.b64[b >> 2];
t = (b & 3) << 4;
j = 1;
break;
case 1:
chunk[i++] = this.b64[t | b >> 4];
t = (b & 15) << 2;
j = 2;
break;
case 2:
chunk[i++] = this.b64[t | b >> 6];
chunk[i++] = this.b64[b & 63];
j = 0;
break;
}
if (i > 8191) {
(parts || (parts = [])).push(String.fromCharCode.apply(String, chunk));
i = 0;
}
}
if (j) {
chunk[i++] = this.b64[t];
chunk[i++] = 61;
if (j === 1)
chunk[i++] = 61;
}
if (parts) {
if (i)
parts.push(String.fromCharCode.apply(String, chunk.slice(0, i)));
return parts.join("");
}
return String.fromCharCode.apply(String, chunk.slice(0, i));
};
/**
* Decodes a base64 encoded string to a buffer.
* @param {string} string Source string
* @param {DataView} buffer Destination buffer
* @param {number} offset Destination offset
* @returns {number} Number of bytes written
* @throws {Error} If encoding is invalid
*/
write(string: string, buffer: DataView, offset: number): number {
var start = offset;
var j = 0, // goto index
t; // temporary
for (var i = 0; i < string.length;) {
var c = string.charCodeAt(i++);
if (c === 61 && j > 1)
break;
if ((c = this.s64[c]) === undefined)
throw Error(this.invalidEncoding);
switch (j) {
case 0:
t = c;
j = 1;
break;
case 1:
buffer.setUint8(offset++, t << 2 | (c & 48) >> 4);
t = c;
j = 2;
break;
case 2:
buffer.setUint8(offset++, (t & 15) << 4 | (c & 60) >> 2);
t = c;
j = 3;
break;
case 3:
buffer.setUint8(offset++, (t & 3) << 6 | c);
j = 0;
break;
}
}
if (j === 1)
throw Error(this.invalidEncoding);
return offset - start;
};
}
const base64 = new Base64();
/**
* A minimal UTF8 implementation for number arrays.
* @memberof util
* @namespace
*/
class UTF8 {
/**
* Calculates the UTF8 byte length of a string.
*/
length(string: string): number {
var len = 0,
c = 0;
for (var i = 0; i < string.length; ++i) {
c = string.charCodeAt(i);
if (c < 128)
len += 1;
else if (c < 2048)
len += 2;
else if ((c & 0xFC00) === 0xD800 && (string.charCodeAt(i + 1) & 0xFC00) === 0xDC00) {
++i;
len += 4;
} else
len += 3;
}
return len;
};
/**
* Reads UTF8 bytes as a string.
*/
read(buffer: DataView, start: number, end: number): string {
var len = end - start;
if (len < 1)
return "";
var parts = null,
chunk = [],
i = 0, // char offset
t; // temporary
while (start < end) {
t = buffer.getUint8(start++);
if (t < 128)
chunk[i++] = t;
else if (t > 191 && t < 224)
chunk[i++] = (t & 31) << 6 | buffer.getUint8(start++) & 63;
else if (t > 239 && t < 365) {
t = ((t & 7) << 18 | (buffer.getUint8(start++) & 63) << 12 | (buffer.getUint8(start++) & 63) << 6 | buffer.getUint8(start++) & 63) - 0x10000;
chunk[i++] = 0xD800 + (t >> 10);
chunk[i++] = 0xDC00 + (t & 1023);
} else
chunk[i++] = (t & 15) << 12 | (buffer.getUint8(start++) & 63) << 6 | buffer.getUint8(start++) & 63;
if (i > 8191) {
(parts || (parts = [])).push(String.fromCharCode.apply(String, chunk));
i = 0;
}
}
if (parts) {
if (i)
parts.push(String.fromCharCode.apply(String, chunk.slice(0, i)));
return parts.join("");
}
return String.fromCharCode.apply(String, chunk.slice(0, i));
};
/**
* Writes a string as UTF8 bytes.
*/
write(string: string, buffer: DataView, offset: number): number {
var start = offset,
c1, // character 1
c2; // character 2
for (var i = 0; i < string.length; ++i) {
c1 = string.charCodeAt(i);
if (c1 < 128) {
buffer.setUint8(offset++, c1);
} else if (c1 < 2048) {
buffer.setUint8(offset++, c1 >> 6 | 192);
buffer.setUint8(offset++, c1 & 63 | 128);
} else if ((c1 & 0xFC00) === 0xD800 && ((c2 = string.charCodeAt(i + 1)) & 0xFC00) === 0xDC00) {
c1 = 0x10000 + ((c1 & 0x03FF) << 10) + (c2 & 0x03FF);
++i;
buffer.setUint8(offset++, c1 >> 18 | 240);
buffer.setUint8(offset++, c1 >> 12 & 63 | 128);
buffer.setUint8(offset++, c1 >> 6 & 63 | 128);
buffer.setUint8(offset++, c1 & 63 | 128);
} else {
buffer.setUint8(offset++, c1 >> 12 | 224);
buffer.setUint8(offset++, c1 >> 6 & 63 | 128);
buffer.setUint8(offset++, c1 & 63 | 128);
}
}
return offset - start;
};
}
const utf8 = new UTF8();
class Encode {
private buffer: ArrayBuffer = null;
private view: DataView = null;
private index: number = 0;
constructor(length: number) {
this.buffer = new ArrayBuffer(length)
this.view = new DataView(this.buffer);
this.index = 0;
}
Int8(data: number) {
if (!isNumber(data)) data = 0;
return this.view.setInt8(this.index++, data);
}
Uint8(data: number) {
if (!isNumber(data)) data = 0;
return this.view.setUint8(this.index++, data);
}
Int16(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setInt16(this.index, data);
this.index += 2;
return value;
}
Uint16(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setUint16(this.index, data);
this.index += 2;
return value;
}
Int32(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setInt32(this.index, data);
this.index += 4;
return value;
}
Uint32(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setUint32(this.index, data);
this.index += 4;
return value;
}
Float32(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setFloat32(this.index, data);
this.index += 4;
return value;
}
Float64(data: number) {
if (!isNumber(data)) data = 0;
var value = this.view.setFloat64(this.index, data);
this.index += 8;
return value;
}
Boolean(data) {
return this.Uint8(data ? 1 : 0);
}
String(string) {
if (!isString(string)) string = '';
const len = utf8.write(string, this.view, this.index + 2);
this.Uint16(len);
this.index += len;
}
Base64(string) {
if (!isBase64(string)) string = '';
const len = base64.write(string, this.view, this.index + 2);
this.Uint16(len);
this.index += len;
}
Array(array) {
if (isArray(array) && !isEmpty(array)) {
return this.String(JSON.stringify(array));
} else {
return this.String('');
}
}
Object(obj) {
if (isMap(obj) && !isEmpty(obj)) {
return this.String(JSON.stringify(obj));
} else {
return this.String('');
}
}
Buffer() {
return this.buffer;
}
}
class Decode {
private view: DataView = null;
private index: number = 0;
constructor(buffer: ArrayBuffer) {
this.view = new DataView(buffer);
this.index = 0;
}
Int8() {
return this.view.getInt8(this.index++);
}
Uint8() {
return this.view.getUint8(this.index++);
}
Int16() {
const value = this.view.getInt16(this.index);
this.index += 2;
return value;
}
Uint16() {
const value = this.view.getUint16(this.index);
this.index += 2;
return value;
}
Int32() {
const value = this.view.getInt32(this.index);
this.index += 4;
return value;
}
Uint32() {
const value = this.view.getUint32(this.index);
this.index += 4;
return value;
}
Float32() {
const value = this.view.getFloat32(this.index);
this.index += 4;
return value;
}
Float64() {
const value = this.view.getFloat64(this.index);
this.index += 8;
return value;
}
Boolean() {
return !!this.Uint8();
}
String() {
const len = this.Uint16();
this.index += len;
return utf8.read(this.view, this.index - len, this.index);
}
Base64() {
const len = this.Uint16();
this.index += len;
return base64.read(this.view, this.index - len, this.index);
}
Array() {
const str = this.String();
return str ? JSON.parse(str) : [];
}
Object() {
const str = this.String();
return str ? JSON.parse(str) : {};
}
}
const getType = function (param) {
return Object.prototype.toString.call(param).slice(8, -1).toLowerCase();
}
const isObject = function (param) {
return param && typeof param === 'object';
}
const isArray = function (param) {
return getType(param) === 'array';
}
const isMap = function (param) {
return getType(param) === 'object';
}
const isString = function (param) {
return getType(param) === 'string';
}
const isNumber = function (param) {
return getType(param) === 'number';
}
const isBoolean = function (param) {
return getType(param) === 'boolean';
}
const isBase64 = function (param) {
return isString(param) && /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(param);
}
function stringStartsWith(str1: string, str2: string) {
if (str1 === str2) {
return true;
}
for (let index = 0; index < str2.length; index++) {
if (str1[index] !== str2[index]) {
return false;
}
}
return true;
}
function isEmpty(obj) {
if (isArray(obj)) {
return !obj.length;
} else if (isMap(obj)) {
for (const key in obj) {
return false;
}
}
return true;
}
function compareStr(str1: string, str2: string) {
if (str1 === str2) {
return 0;
}
if (str1.length > str2.length) {
return 1;
}
if (str1.length < str2.length) {
return -1;
}
for (let i = 0, code1 = 0, code2 = 0; i < str1.length; i++) {
if (str2.length <= i) {
return 1;
} else {
code1 = str1.charCodeAt(i);
code2 = str2.charCodeAt(i);
if (code1 > code2) {
return 1;
} else if (code1 < code2) {
return -1;
}
}
}
return 0;
}
function sortKeys(obj) {
if (isMap(obj)) {
let index = 0;
const keys: string[] = [];
for (const key in obj) {
for (index = keys.length - 1; index >= 0; index--) {
if (compareStr(key, keys[index]) >= 0) {
break;
}
}
if (index === keys.length - 1) {
keys.push(key);
} else {
keys.splice(index + 1, 0, key);
}
}
return keys;
} else if (isArray(obj)) {
return obj.map(function (v, k) {
return k;
})
}
return [];
}
function realType(type) {
if (isObject(type)) {
return type;
}
return protoCache[type] || type;
}
const singleArrayPrefix = 'SingleArray';
function isSingleArray(str: string) {
return isString(str) && stringStartsWith(str, singleArrayPrefix);
}
function SingleArrayProto(str: string) {
const stringify = str.slice(singleArrayPrefix.length + 1, -1);
return JSON.parse(stringify);
}
/**
* 标记单一类型的数组
* @param proto
*/
export const singleArray = function (proto) {
return `${singleArrayPrefix}(${JSON.stringify(proto)})`;
}
function DataLen(data: any, proto: any) {
proto = realType(proto);
let length = 0;
if (isMap(proto)) {
if (!isMap(data)) data = {};
for (const key in proto) {
length += DataLen(data[key], proto[key]);
}
} else if (isArray(proto)) {
if (!isArray(data)) data = [];
proto.forEach(function (type, index) {
length += DataLen(data[index], type);
})
} else if (proto === 'String') {
// 如果是String的话，固定开头有2字节记录字符串长度
length += 2;
if (isString(data)) length += utf8.length(data);
} else if (proto === 'Object' || proto === 'Array') {
// Object和Array类型也会将数据通过JSON.stringify转成String格式
length += 2;
if (!isEmpty(data)) length += utf8.length(JSON.stringify(data));
} else if (proto === 'Base64') {
// 如果是Base64的话，固定开头有2字节记录字符串长度
length += 2;
if (isBase64(data)) length += base64.length(data);
} else if (isSingleArray(proto)) {
// 如果是SingleArray的话，固定开头有2字节记录数组长度
length += 2;
if (!isArray(data)) data = [];
proto = realType(SingleArrayProto(proto));
data.forEach(function (value) {
length += DataLen(value, proto);
})
} else if (TypeByte[proto]) {
length += TypeByte[proto];
} else {
throw new Error("'proto' is bad");
}
return length;
}
function encodeData(encode: Encode, data: any, proto: any) {
proto = realType(proto);
if (isMap(proto)) {
if (!isMap(data)) data = {};
sortKeys(proto).forEach(function (key) {
encodeData(encode, data[key], proto[key]);
})
} else if (isArray(proto)) {
if (!isArray(data)) data = [];
proto.forEach(function (type, index) {
encodeData(encode, data[index], type);
})
} else if (isSingleArray(proto)) {
if (!isArray(data)) data = [];
encode.Uint16(data.length);
proto = realType(SingleArrayProto(proto));
data.forEach(function (value) {
encodeData(encode, value, proto);
})
} else {
encode[proto](data);
}
}
function decodeData(decode: Decode, proto: any) {
proto = realType(proto);
if (isMap(proto)) {
const obj = {};
sortKeys(proto).forEach(function (key) {
obj[key] = decodeData(decode, proto[key]);
});
return obj;
} else if (isArray(proto)) {
return proto.map(function (type) {
return decodeData(decode, type);
});
} else if (isSingleArray(proto)) {
const arr = [];
const len = decode.Uint16();
proto = realType(SingleArrayProto(proto));
for (let index = 0; index < len; index++) {
arr.push(decodeData(decode, proto));
}
return arr;
} else {
return decode[proto]();
}
}
const TypeByte = {
'Int8': 1,
'Uint8': 1,
'Int16': 2,
'Uint16': 2,
'Int32': 4,
'Uint32': 4,
'Float32': 4,
'Float64': 8,
'BigInt64': 8,
'BigUint64': 8,
'Boolean': 1,
'String': 1,
'Base64': 1,
'Array': 1,
'Object': 1
}
export const Type = {
'Int8': 'Int8', // 1byte -128 to 127
'Uint8': 'Uint8', // 1byte 0 to 255
'Uint8Clamped': 'Uint8', // 1byte 0 to 255
'Int16': 'Int16', // 2byte -32768 to 32767
'Uint16': 'Uint16', // 2byte 0 to 65535
'Int32': 'Int32', // 4byte -2147483648 to 2147483647
'Uint32': 'Uint32', // 4byte 0 to 4294967295
'Float32': 'Float32', // 4byte 1.2x10^-38 to 3.4x10^38
'Float64': 'Float64', // 8byte 5.0x10^-324 to 1.8x10^308
'BigInt64': 'BigInt64', // 8byte -2^63 to (2^63)-1
'BigUint64': 'BigUint64', // 8byte 0 to (2^64)-1
'Boolean': 'Boolean', // 1byte 0 to 255
'String': 'String', // 1byte 0 to 255
'Base64': 'Base64', // 1byte 0 to 255
'Array': 'Array', // 1byte 0 to 255
'Object': 'Object' // 1byte 0 to 255
}
/**
* 序列化
* 开头2字节用来存储proto的id
*/
export const encode = function (obj: Object, id: number | string) {
const proto = protoCache[id];
if (proto) {
const len = DataLen(obj, proto);
const encode = new Encode(len + 2);
encode.Uint16(Number(id));
encodeData(encode, obj, proto);
return encode.Buffer();
} else {
throw new Error("encode error: 'id' is bad");
}
}
/**
* 反序列化
* 开头2字节代表proto的id
*/
export const decode = function (buffer: ArrayBuffer) {
const decode = new Decode(buffer);
const id = decode.Uint16();
const proto = protoCache[id];
if (proto) {
return decodeData(decode, proto);
} else {
throw new Error("decode error: 'buffer' is bad");
}
}
/**
* proto缓存
*/
const protoCache = {}
/**
* 注册proto
* id: 必须是个正整数(或正整数字符串), 取值范围[0,65535]
*/
export const registerProto = function (id: number | string, proto: any) {
if (typeof id === 'string') id = Number(id);
if (isNumber(id) && Math.floor(id) === id && id >= 0 && id <= 65535 && !Type[id]) {
protoCache[id] = proto;
} else {
throw new Error("registerProto error: 'id' is bad");
}
}
export const registerProtoMap = function (protoMap: any) {
if (isMap(protoMap)) {
for (const id in protoMap) {
registerProto(id, protoMap[id]);
}
} else {
throw new Error("registerProtoMap error: 'protoMap' is bad");
}
}
export const protoToJson = function () {
return JSON.stringify(protoCache);
}
2赞
对，是1字节
，我都迷糊了
真心觉得没必要重复造轮子
我觉得protobuf的精髓在于协议本身，而不是代码
一个功能的.proto协议约定之后，可以说整个功能就基本成型了，客户端服务器可以完全独立开发，剩下就是工作量的问题
你这个库可以说是在没有protobuf之前常用的二进制消息的简单封装，反而是退步了，你可以吧registerProto独立成一个解析器，出一个类似.proto文件的协议格式
再说下两点不足：
1，没有约定大小端，这个在跟服务器通信中是必须约定的
2，服务器限定node.js，其他流行的服务器语言（C++、Go）就用不了了
1赞
我们也是这么做的
求教大小端是什么？
其实没有限定nodejs，其它语言自己适配就好了。
我造轮子其中一点就是不习惯protobuffer的协议，我现在这个的协议是json格式的，更符合js的使用
造轮子有几点把：
1、protobuffer在前端太难用了(两年前是够难用的，现在不知道)，存在一定使用成本，而且由于在小程序上没法动态new Function，它和后端在性能等方面区别应该很大。
2、protobuffer的协议存在学习成本，就那个proto我看着就很难受。
3、还有一个最重要的就是，丰富自己的知识储备，不动手我怕永远也不够了解它。
其实这个至于好不好用我也不知道，毕竟连我自己都没用，不过轮子我还是会继续造的。
感谢你的回复
大端：数据的高字节在内存低地址， 小端：数据的高字节在内存高地址。以0x1234这个整数为例 ， 大端的话，12的地址比34低， 小端则12的地址比34高。反正我们用的 是大端格式。
其实用啥端无所谓，只要“包装”和“拆包”的协议一致，就行了。