愿星光伴随你左右


  • 首页

  • todo

  • 思考

  • life

  • food

  • OS

  • lua

  • redis

  • Golang

  • C

  • TCP/IP

  • ebpf

  • p4

  • OpenVPN

  • IPSec

  • L2TP

  • DNS

  • distributed

  • web

  • OpenWRT

  • 运维

  • Git

  • 鸟哥的私房菜

  • IT杂谈

  • 投资

  • About Me

  • 友情链接

  • FTP

  • 搜索
close

数据库事务的四大原则

时间: 2021-08-18   |   分类: db     |   阅读: 3639 字 ~8分钟

数据库的事务四大原则

说到数据库,以前我老师有一句很经典的话。你可以不会写SQL,但是一定不能不知道ACID。

在工业领域,SQL可以说是应用最广泛的技术。从后端到算法,从数据到DBA,再到产品,甚至连一些运营也会基本的SQL。所以如果你现在还不太会的话,我建议你用一个下午的时间找个网站好好学一下。

阅读全文 »

详解Redis源码中的部分快速排序算法pqsort

时间: 2021-08-16   |   分类: redis     |   阅读: 5775 字 ~12分钟

详解Redis源码中的部分快速排序算法(pqsort.c)

转载自果冻虾仁 2015-06-07 19:08:39


  看标题,你可能会疑惑:咦?你这家伙,怎么不讲解完整的快排,只讲一部分快排……-。- 哎,冤枉。“部分快排”是算法的名字,实际上本文相当详细呢。本文几乎与普通快排无异。看懂了本文,你对普通的快排也会有更深的认识了。

阅读全文 »

一键搞定 VSCode 下的 C/C++基本开发环境配置

时间: 2021-08-13   |   分类: 运维     |   阅读: 2588 字 ~6分钟

以下内容转载自 https://zhuanlan.zhihu.com/p/137760796

如果你能看懂并确认已做到以下两点:

  1. C/C++编译器(例如MingGW)环境已经配置好
  2. VSCode已安装C/C++插件

那就废话不多说, 直接开始:

阅读全文 »

常见的10种排序算法

时间: 2021-08-12   |   分类: algorithm     |   阅读: 7552 字 ~16分钟

[常见的排序算法——常见的10种排序]

常见算法可以分为两大类:

  非线性时间比较类排序:通过比较来决定元素间的相对次序,由于其时间复杂度不能突破O(nlogn),因此称为非线性时间比较类排序。

  线性时间非比较类排序:不通过比较来决定元素间的相对次序,它可以突破基于比较排序的时间下界,以线性时间运行,因此称为线性时间非比较类排序。 图片

阅读全文 »

redis-lzf压缩算法

时间: 2021-08-10   |   分类: redis     |   阅读: 894 字 ~2分钟

文件

lzf.h lzfP.h lzf_c.c (压缩) lzf_d.c (解压)

压缩

  • 默认模式是VERY_FAST

  • 核心思想

    • 对重复值进行压缩
    • 通过hash表来判断是否重复数据
  • 三种模式

模式 (压缩)时间 (压缩后)空间
ULTRA_FAST 极端快 大
VERY_FAST 非常快 中
普通 快 小
  • 变量

LZF_HSLOT_BIAS 64位系统,是in_data的起始地址 ip 输入游标 op 输出游标 hval 整型,当前游标的4位数值ip[-1]|ip[0]|ip[1]|ip[2] htab 保存输入数据(in_data)的下标 hslot 哈希槽,指向哈希表(htab)的指针 ref 值引用,ref = *hslot + LZF_HSLOT_BIAS,也是引用in_data的数据 off 偏移量,ip相对于ref的偏移量 lit literal,非压缩字符串长度

阅读全文 »

redis基础组件-ae简介

时间: 2021-08-10   |   分类: redis     |   阅读: 776 字 ~2分钟

数据结构

/* File event structure */
typedef struct aeFileEvent {
    int mask; /* one of AE_(READABLE|WRITABLE) */
    aeFileProc *rfileProc;
    aeFileProc *wfileProc;
    void *clientData;
} aeFileEvent;

/* Time event structure */
typedef struct aeTimeEvent {
    long long id; /* time event identifier. */
    long when_sec; /* seconds */
    long when_ms; /* milliseconds */
    aeTimeProc *timeProc;
    aeEventFinalizerProc *finalizerProc;
    void *clientData;
    struct aeTimeEvent *next;
} aeTimeEvent;

/* A fired event */
typedef struct aeFiredEvent {
    int fd;
    int mask;
} aeFiredEvent;

/* State of an event based program */
typedef struct aeEventLoop {
    int maxfd;
    long long timeEventNextId;
    aeFileEvent events[AE_SETSIZE]; /* Registered events */
    aeFiredEvent fired[AE_SETSIZE]; /* Fired events */
    aeTimeEvent *timeEventHead;
    int stop;
    void *apidata; /* This is used for polling API specific data */
    aeBeforeSleepProc *beforesleep;
} aeEventLoop;

特点

  • 整个redis服务器运行的发动机
  • 利用OS提供的多路复用机制,高效的处理句柄的读写事件和超时事件
  • 同一个fd,可读和可写的event是分开的(fd可能随时来数据,所以时刻监听其可读,但只有在还有待写数据的情况下才需要监听fd的可写事件)
  • 在处理时间事件的句柄中,不处理新生成的时间事件

重要接口函数

  • 处理文件fd的读写事件,若没有则等待最近的时间事件超时
/* Process every pending time event, then every pending file event
 * (that may be registered by time event callbacks just processed).
 * Without special flags the function sleeps until some file event
 * fires, or when the next time event occurrs (if any).
 *
 * If flags is 0, the function does nothing and returns.
 * if flags has AE_ALL_EVENTS set, all the kind of events are processed.
 * if flags has AE_FILE_EVENTS set, file events are processed.
 * if flags has AE_TIME_EVENTS set, time events are processed.
 * if flags has AE_DONT_WAIT set the function returns ASAP until all
 * the events that's possible to process without to wait are processed.
 *
 * The function returns the number of events processed. */
int aeProcessEvents(aeEventLoop *eventLoop, int flags)
{
    int processed = 0, numevents;

    /* Nothing to do? return ASAP */
    if (!(flags & AE_TIME_EVENTS) && !(flags & AE_FILE_EVENTS)) return 0;

    /* Note that we want call select() even if there are no
     * file events to process as long as we want to process time
     * events, in order to sleep until the next time event is ready
     * to fire. */
    if (eventLoop->maxfd != -1 ||
        ((flags & AE_TIME_EVENTS) && !(flags & AE_DONT_WAIT))) {
        int j;
        aeTimeEvent *shortest = NULL;
        struct timeval tv, *tvp;

        if (flags & AE_TIME_EVENTS && !(flags & AE_DONT_WAIT))
            shortest = aeSearchNearestTimer(eventLoop);
        if (shortest) {
            long now_sec, now_ms;

            /* Calculate the time missing for the nearest
             * timer to fire. */
            aeGetTime(&now_sec, &now_ms);
            tvp = &tv;
            tvp->tv_sec = shortest->when_sec - now_sec;
            if (shortest->when_ms < now_ms) {
                tvp->tv_usec = ((shortest->when_ms+1000) - now_ms)*1000;
                tvp->tv_sec --;
            } else {
                tvp->tv_usec = (shortest->when_ms - now_ms)*1000;
            }
            if (tvp->tv_sec < 0) tvp->tv_sec = 0;
            if (tvp->tv_usec < 0) tvp->tv_usec = 0;
        } else {
            /* If we have to check for events but need to return
             * ASAP because of AE_DONT_WAIT we need to se the timeout
             * to zero */
            if (flags & AE_DONT_WAIT) {
                tv.tv_sec = tv.tv_usec = 0;
                tvp = &tv;
            } else {
                /* Otherwise we can block */
                tvp = NULL; /* wait forever */
            }
        }

        numevents = aeApiPoll(eventLoop, tvp);
        for (j = 0; j < numevents; j++) {
            aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
            int mask = eventLoop->fired[j].mask;
            int fd = eventLoop->fired[j].fd;
            int rfired = 0;

	    /* note the fe->mask & mask & ... code: maybe an already processed
             * event removed an element that fired and we still didn't
             * processed, so we check if the event is still valid. */
            if (fe->mask & mask & AE_READABLE) {
                rfired = 1;
                fe->rfileProc(eventLoop,fd,fe->clientData,mask);
            }
            if (fe->mask & mask & AE_WRITABLE) {
                if (!rfired || fe->wfileProc != fe->rfileProc)
                    fe->wfileProc(eventLoop,fd,fe->clientData,mask);
            }
            processed++;
        }
    }
    /* Check time events */
    if (flags & AE_TIME_EVENTS)
        processed += processTimeEvents(eventLoop);

    return processed; /* return the number of processed file/time events */
}

redis基础组件-anet简介

时间: 2021-08-10   |   分类: redis     |   阅读: 444 字 ~1分钟

特点

  • 模块非常简洁,提供了监听指定端口,链接指定地址,读,写网络句柄等功能

重要的接口函数

  • 监听指定端口
int anetTcpServer(char *err, int port, char *bindaddr)
{
    int s, on = 1;
    struct sockaddr_in sa;
    
    if ((s = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        anetSetError(err, "socket: %s\n", strerror(errno));
        return ANET_ERR;
    }
    if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) {
        anetSetError(err, "setsockopt SO_REUSEADDR: %s\n", strerror(errno));
        close(s);
        return ANET_ERR;
    }
    memset(&sa,0,sizeof(sa));
    sa.sin_family = AF_INET;
    sa.sin_port = htons(port);
    sa.sin_addr.s_addr = htonl(INADDR_ANY);
    if (bindaddr) {
        if (inet_aton(bindaddr, &sa.sin_addr) == 0) {
            anetSetError(err, "Invalid bind address\n");
            close(s);
            return ANET_ERR;
        }
    }
    if (bind(s, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
        anetSetError(err, "bind: %s\n", strerror(errno));
        close(s);
        return ANET_ERR;
    }
    if (listen(s, 511) == -1) { /* the magic 511 constant is from nginx */
        anetSetError(err, "listen: %s\n", strerror(errno));
        close(s);
        return ANET_ERR;
    }
    return s;
}
  • 链接指定地址
static int anetTcpGenericConnect(char *err, char *addr, int port, int flags)
{
    int s, on = 1;
    struct sockaddr_in sa;

    if ((s = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        anetSetError(err, "creating socket: %s\n", strerror(errno));
        return ANET_ERR;
    }
    /* Make sure connection-intensive things like the redis benckmark
     * will be able to close/open sockets a zillion of times */
    setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on));

    sa.sin_family = AF_INET;
    sa.sin_port = htons(port);
    if (inet_aton(addr, &sa.sin_addr) == 0) {
        struct hostent *he;

        he = gethostbyname(addr);
        if (he == NULL) {
            anetSetError(err, "can't resolve: %s\n", addr);
            close(s);
            return ANET_ERR;
        }
        memcpy(&sa.sin_addr, he->h_addr, sizeof(struct in_addr));
    }
    if (flags & ANET_CONNECT_NONBLOCK) {
        if (anetNonBlock(err,s) != ANET_OK)
            return ANET_ERR;
    }
    if (connect(s, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
        if (errno == EINPROGRESS &&
            flags & ANET_CONNECT_NONBLOCK)
            return s;

        anetSetError(err, "connect: %s\n", strerror(errno));
        close(s);
        return ANET_ERR;
    }
    return s;
}
  • 读写指定句柄
/* Like read(2) but make sure 'count' is read before to return
 * (unless error or EOF condition is encountered) */
int anetRead(int fd, char *buf, int count)
{
    int nread, totlen = 0;
    while(totlen != count) {
        nread = read(fd,buf,count-totlen);
        if (nread == 0) return totlen;
        if (nread == -1) return -1;
        totlen += nread;
        buf += nread;
    }
    return totlen;
}

/* Like write(2) but make sure 'count' is read before to return
 * (unless error is encountered) */
int anetWrite(int fd, char *buf, int count)
{
    int nwritten, totlen = 0;
    while(totlen != count) {
        nwritten = write(fd,buf,count-totlen);
        if (nwritten == 0) return totlen;
        if (nwritten == -1) return -1;
        totlen += nwritten;
        buf += nwritten;
    }
    return totlen;
}

redis基础组件-zmalloc简介

时间: 2021-08-10   |   分类: redis     |   阅读: 228 字 ~1分钟

数据结构

------------
len|context
------------

特点

  • 申请内存时,自带整个MEM的len,用于统计当前进程总共消耗的MEM

模糊点

  • 在申请内存的接口中怎么确定ptr+sizeof(size_t)后的地址一定就能满足内存对齐?

主要接口函数

  • 申请一片内存
void *zmalloc(size_t size) {
    void *ptr = malloc(size+PREFIX_SIZE);

    if (!ptr) zmalloc_oom(size);
#ifdef HAVE_MALLOC_SIZE
    increment_used_memory(redis_malloc_size(ptr));
    return ptr;
#else
    *((size_t*)ptr) = size;
    increment_used_memory(size+PREFIX_SIZE);
    return (char*)ptr+PREFIX_SIZE;
#endif
}
  • 调整内存大小
void *zrealloc(void *ptr, size_t size) {
#ifndef HAVE_MALLOC_SIZE
    void *realptr;
#endif
    size_t oldsize;
    void *newptr;

    if (ptr == NULL) return zmalloc(size);
#ifdef HAVE_MALLOC_SIZE
    oldsize = redis_malloc_size(ptr);
    newptr = realloc(ptr,size);
    if (!newptr) zmalloc_oom(size);

    decrement_used_memory(oldsize);
    increment_used_memory(redis_malloc_size(newptr));
    return newptr;
#else
    realptr = (char*)ptr-PREFIX_SIZE;
    oldsize = *((size_t*)realptr);
    newptr = realloc(realptr,size+PREFIX_SIZE);
    if (!newptr) zmalloc_oom(size);

    *((size_t*)newptr) = size;
    decrement_used_memory(oldsize);
    increment_used_memory(size);
    return (char*)newptr+PREFIX_SIZE;
#endif
}
  • 释放内存
void zfree(void *ptr) {
#ifndef HAVE_MALLOC_SIZE
    void *realptr;
    size_t oldsize;
#endif

    if (ptr == NULL) return;
#ifdef HAVE_MALLOC_SIZE
    decrement_used_memory(redis_malloc_size(ptr));
    free(ptr);
#else
    realptr = (char*)ptr-PREFIX_SIZE;
    oldsize = *((size_t*)realptr);
    decrement_used_memory(oldsize+PREFIX_SIZE);
    free(realptr);
#endif
}
53 54 55 56 57 58 59 60 61

日志
分类
标签
RSS 订阅
GitHub
© 2009 - 2025
粤ICP备2021068940号-1 粤公网安备44011302003059
0%