Android 浅析 Ashmem

发表于   |  

前言

Linus Benedict Torvalds : RTFSC – Read The Funning Source Code

概括

匿名共享内存子系统Ashmem（Anonymous Shared Memory），它以驱动程序的形式实现在内核空间中。它有两个特点，一是能够辅助内存管理系统来有效地管理不再使用的内存块，二是它通过Binder进程间通信机制来实现进程间的内存共享。
Android的Ashmem建立在Linux内核实现的共享内存的基础上封装了一层。

Java 接口分析

Android应用程序框架层，提供了一个MemoryFile接口来封装了匿名共享内存文件的创建和使用

地址：frameworks/base/core/java/android/os/MemoryFile.java

首先通过底层接口来大致了解下ashmem的操作：

1
2
3
4
5
6
7
8
static struct file_operations ashmem_fops = {  
    .owner = THIS_MODULE,  
    .open = ashmem_open,  
    .release = ashmem_release,  
    .mmap = ashmem_mmap,  
    .unlocked_ioctl = ashmem_ioctl,  
    .compat_ioctl = ashmem_ioctl,  
};

它提供了open、mmap、release和ioctl四种操作，因为是通过内存映射地址来进行读写的操作，所以没有提供read和write操作。

构造函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public MemoryFile(String name, int length) throws IOException {
    mLength = length;
    if (length >= 0) {
        mFD = native_open(name, length);
    } else {
        throw new IOException("Invalid length: " + length);
    }
    if (length > 0) {
        mAddress = native_mmap(mFD, length, PROT_READ | PROT_WRITE);
    } else {
        mAddress = 0;
    }
}

构造函数具体的工作就是通过JNI调用C++的内核来进行初始化的工作，具体就是实现了open和mmap的构造。

  • native_open：主要执行的操作是调用open打开设备文件/dev/ashmem，在Ashmem驱动程序中创建了一个ashmem_area结构，表示一块新的共享内存，接着调用了两次ioctl文件操作分别来设置这块新建的匿名共享内存的名字和大小。
  • native_mmap：首先获取文件描述符fd，fd是在前面open匿名设备文件/dev/ashmem获得的，之后通过mmap来执行内存映射操作了。最后的调用函数是ashmem_mmap，ashmem_mmap调用了Linux内核提供的shmem_file_setup函数来在临时文件系统tmpfs中创建一个临时文件，这个临时文件与Ashmem驱动程序创建的匿名共享内存对应。函数shmem_file_setup是Linux内核中用来创建共享内存文件的方法，而Linux内核中的共享内存机制其实是一种进程间通信（IPC）机制，它的实现相对也是比较复杂，Android系统的匿名共享内存机制正是由于直接使用了Linux内核共享内存机制，才会很小巧。最后，ashmem_mmap执行完成后，经过层层返回到JNI方法native_mmap中去，就从mmap函数的返回值中得到了这块虚拟空间的起始地址，这个起始地址最终返回到应用程序框架层的MemoryFile类的构造函数中，并且保存在成员变量mAddress中，接下来共享内存的读写操作就是对这个地址空间进行操作了。

读写函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
private static native int native_read(FileDescriptor fd, long address, byte[] buffer, int srcOffset, int destOffset, int count, boolean isUnpinned) throws IOException;
private static native void native_write(FileDescriptor fd, long address, byte[] buffer, int srcOffset, int destOffset, int count, boolean isUnpinned) throws IOException;
/**
 * Reads bytes from the memory file.
 * Will throw an IOException if the file has been purged.
 */
public int readBytes(byte[] buffer, int srcOffset, int destOffset, int count) throws IOException {
    ...
    return native_read(mFD, mAddress, buffer, srcOffset, destOffset, count, mAllowPurging);
}
/**
 * Write bytes to the memory file.
 * Will throw an IOException if the file has been purged.
 */
public void writeBytes(byte[] buffer, int srcOffset, int destOffset, int count) throws IOException {
    ...
    native_write(mFD, mAddress, buffer, srcOffset, destOffset, count, mAllowPurging);
}

address参数就是我们在前面执行mmap来映射匿名共享内存文件到内存中时，得到的进程虚拟地址空间的起始地址了,接着就能够直接访问了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
static jint android_os_MemoryFile_read(JNIEnv* env, jobject clazz, jobject fileDescriptor, jint address, jbyteArray buffer, jint srcOffset, jint destOffset, jint count, jboolean unpinned)  
{  
    int fd = jniGetFDFromFileDescriptor(env, fileDescriptor);  
    if (unpinned && ashmem_pin_region(fd, 0, 0) == ASHMEM_WAS_PURGED) {  
        ashmem_unpin_region(fd, 0, 0);  
        jniThrowException(env, "java/io/IOException", "ashmem region was purged");  
        return -1;  
    }  
  
    env->SetByteArrayRegion(buffer, destOffset, count, (const jbyte *)address + srcOffset);  
  
    if (unpinned) {  
        ashmem_unpin_region(fd, 0, 0);  
    }  
    return count;  
}  
  
static jint android_os_MemoryFile_write(JNIEnv* env, jobject clazz, jobject fileDescriptor, jint address, jbyteArray buffer, jint srcOffset, jint destOffset, jint count, jboolean unpinned)  
{  
    int fd = jniGetFDFromFileDescriptor(env, fileDescriptor);  
    if (unpinned && ashmem_pin_region(fd, 0, 0) == ASHMEM_WAS_PURGED) {  
        ashmem_unpin_region(fd, 0, 0);  
        jniThrowException(env, "java/io/IOException", "ashmem region was purged");  
        return -1;  
    }  
  
    env->GetByteArrayRegion(buffer, srcOffset, count, (jbyte *)address + destOffset);  
  
    if (unpinned) {  
        ashmem_unpin_region(fd, 0, 0);  
    }  
    return count;  
}

这里是内核具体的读写操作，我们可以看到在拿到内存首地址后对数据的访问都是直接的。

加锁和解锁函数

Android系统的运行时库提到了执行匿名共享内存的锁定和解锁操作的两个函数ashmem_pin_region和ashmem_unpin_region。

1
2
3
4
5
6
7
8
9
10
11
int ashmem_pin_region(int fd, size_t offset, size_t len)  
{  
    struct ashmem_pin pin = { offset, len };  
    return ioctl(fd, ASHMEM_PIN, &pin);  
}  
  
int ashmem_unpin_region(int fd, size_t offset, size_t len)  
{  
    struct ashmem_pin pin = { offset, len };  
    return ioctl(fd, ASHMEM_UNPIN, &pin);  
}

它们最后都是通过ioctl来进行操作，只是传入的参数各不相同。分别是ASHMEM_PIN和ASHMEM_UNPIN。

首先先来了解下锁的数据结构：

1
2
3
4
5
6
7
8
9
10
11
12
13
/* 
 * ashmem_range - represents an interval of unpinned (evictable) pages 
 * Lifecycle: From unpin to pin 
 * Locking: Protected by `ashmem_mutex' 
 */  
struct ashmem_range {  
    struct list_head lru;       /* entry in LRU list */  
    struct list_head unpinned;  /* entry in its area's unpinned list */  
    struct ashmem_area *asma;   /* associated area */  
    size_t pgstart;         /* starting page, inclusive */  
    size_t pgend;           /* ending page, inclusive */  
    unsigned int purged;        /* ASHMEM_NOT or ASHMEM_WAS_PURGED */  
};

1
2
3
4
5
6
7
8
9
switch (cmd) {  
case ASHMEM_PIN:  
    ret = ashmem_pin(asma, pgstart, pgend); 
    break;  
case ASHMEM_UNPIN:  
    ret = ashmem_unpin(asma, pgstart, pgend);  
    break;  
......  
}

根据当前要执行的是ASHMEM_PIN操作还是ASHMEM_UNPIN操作来分别执行ashmem_pin和ashmem_unpin来进一步处理。创建匿名共享内存时，默认所有的内存都是pinned状态的，只有用户告诉Ashmem驱动程序要unpin某一块内存时，Ashmem驱动程序才会把这块内存unpin，之后，用户可以再告诉Ashmem驱动程序要重新pin某一块之前被unpin过的内块，从而把这块内存从unpinned状态改为pinned状态，也就是说，执行ASHMEM_PIN操作时，目标对象必须是一块当前处于unpinned状态的内存块。

ashmem_unpin函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
/* 
 * ashmem_unpin - unpin the given range of pages. Returns zero on success. 
 * 
 * Caller must hold ashmem_mutex. 
 */  
static int ashmem_unpin(struct ashmem_area *asma, size_t pgstart, size_t pgend)  
{  
    struct ashmem_range *range, *next;  
    unsigned int purged = ASHMEM_NOT_PURGED;  
  
restart:  
    list_for_each_entry_safe(range, next, &asma->unpinned_list, unpinned) {  
        /* short circuit: this is our insertion point */  
        if (range_before_page(range, pgstart))  
            break;  
  
        /* 
         * The user can ask us to unpin pages that are already entirely 
         * or partially pinned. We handle those two cases here. 
         */  
        if (page_range_subsumed_by_range(range, pgstart, pgend))  
            return 0;  
        if (page_range_in_range(range, pgstart, pgend)) {  
            pgstart = min_t(size_t, range->pgstart, pgstart),  
            pgend = max_t(size_t, range->pgend, pgend);  
            purged |= range->purged;  
            range_del(range);  
            goto restart;  
        }  
    }  
  
    return range_alloc(asma, range, purged, pgstart, pgend);  
}

函数的主体就是在遍历asma->unpinned_list列表，从中查找当前处于unpinned状态的内存块是否与将要unpin的内存块[pgstart, pgend]是否相交，如果相交，则要执行合并操作，即调整pgstart和pgend的大小，然后通过调用range_del函数删掉原来的已经被unpinned过的内存块，最后再通过range_alloc函数来重新unpinned这块调整过后的内存块[pgstart, pgend]，这里新的内存块[pgstart, pgend]已经包含了刚才所有被删掉的unpinned状态的内存。注意，这里如果找到一块相并的内存块，并且调整了pgstart和pgend的大小之后，要重新再扫描一遍asma->unpinned_list列表，因为新的内存块[pgstart, pgend]可能还会与前后的处于unpinned状态的内存块发生相交。

ashmem_pin函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
/* 
 * ashmem_pin - pin the given ashmem region, returning whether it was 
 * previously purged (ASHMEM_WAS_PURGED) or not (ASHMEM_NOT_PURGED). 
 * 
 * Caller must hold ashmem_mutex. 
 */  
static int ashmem_pin(struct ashmem_area *asma, size_t pgstart, size_t pgend)  
{  
    struct ashmem_range *range, *next;  
    int ret = ASHMEM_NOT_PURGED;  
  
    list_for_each_entry_safe(range, next, &asma->unpinned_list, unpinned) {  
        /* moved past last applicable page; we can short circuit */  
        if (range_before_page(range, pgstart))  
            break;  
  
        /* 
         * The user can ask us to pin pages that span multiple ranges, 
         * or to pin pages that aren't even unpinned, so this is messy. 
         * 
         * Four cases: 
         * 1. The requested range subsumes an existing range, so we 
         *    just remove the entire matching range. 
         * 2. The requested range overlaps the start of an existing 
         *    range, so we just update that range. 
         * 3. The requested range overlaps the end of an existing 
         *    range, so we just update that range. 
         * 4. The requested range punches a hole in an existing range, 
         *    so we have to update one side of the range and then 
         *    create a new range for the other side. 
         */  
        if (page_range_in_range(range, pgstart, pgend)) {  
            ret |= range->purged;  
  
            /* Case #1: Easy. Just nuke the whole thing. */  
            if (page_range_subsumes_range(range, pgstart, pgend)) {  
                range_del(range);  
                continue;  
            }  
  
            /* Case #2: We overlap from the start, so adjust it */  
            if (range->pgstart >= pgstart) {  
                range_shrink(range, pgend + 1, range->pgend);  
                continue;  
            }  
  
            /* Case #3: We overlap from the rear, so adjust it */  
            if (range->pgend <= pgend) {  
                range_shrink(range, range->pgstart, pgstart-1);  
                continue;  
            }  
  
            /* 
             * Case #4: We eat a chunk out of the middle. A bit 
             * more complicated, we allocate a new range for the 
             * second half and adjust the first chunk's endpoint. 
             */  
            range_alloc(asma, range, range->purged,  
                    pgend + 1, range->pgend);  
            range_shrink(range, range->pgstart, pgstart - 1);  
            break;  
        }  
    }  
  
    return ret;  
}

首先要执行pin操作的内存块必须要在unpinned_list列表中的，如果不在，就什么都不用做。要判断要pin的内存块是否在unpinned_list列表中，还是要通过遍历相应的asma->unpinned_list列表来找出与之相交的内存块了。

辅助内存管理

ashmem_init函数：
ashmem_init函数是Ashmem驱动程序模块初始化函数

1
2
3
4
5
6
7
8
9
10
11
12
static struct shrinker ashmem_shrinker = {  
    .shrink = ashmem_shrink,  
    .seeks = DEFAULT_SEEKS * 4,  
};  
  
static int __init ashmem_init(void)  
{  
    int ret;  
    ......  
    register_shrinker(&ashmem_shrinker);
    return 0;  
}

调用register_shrinker函数向内存管理系统注册一个内存回收算法函数。在Linux内核中，当系统内存紧张时，内存管理系统就会进行内存回收算法，将一些最近没有用过的内存换出物理内存去，这样可以增加物理内存的供应。因此，当内存管理系统进行内存回收时，就会调用到这里的ashmem_shrink函数，让Ashmem驱动程序执行内存回收操作。
具体的管理函数可以在kernel/common/mm/memory.c中找到。

简析Ashmen进程间传递

在Linux系统中，文件描述符其实就是一个整数。每一个进程在内核空间都有一个打开文件的数组，这个文件描述符的整数值就是用来索引这个数组的，而且，这个文件描述符只是在本进程内有效，也就是说，在不同的进程中，相同的文件描述符的值，代表的可能是不同的打开文件。因此，在进程间传输文件描述符时，不能简要地把一个文件描述符从一个进程传给另外一个进程，中间必须做一过转换，使得这个文件描述在目标进程中是有效的，并且它和源进程的文件描述符所对应的打开文件是一致的，这样才能保证共享。

在讲解前了解两个结构体：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
/* 
 * This is the flattened representation of a Binder object for transfer between processes.  The 'offsets' supplied as part of a binder transaction contains offsets into the data where these structures occur.  The Binder  driver takes care of re-writing the structure type and data as it moves  between processes. 
 */  
struct flat_binder_object {  
    /* 8 bytes for large_flat_header. */  
    unsigned long       type;  
    unsigned long       flags;  
  
    /* 8 bytes of data. */  
    union {  
        void        *binder;    /* local object */  
        signed long handle;     /* remote object */  
    };  
  
    /* extra data associated with local object */  
    void            *cookie;  
}; 
enum {  
    BINDER_TYPE_BINDER  = B_PACK_CHARS('s', 'b', '*', B_TYPE_LARGE),  
    BINDER_TYPE_WEAK_BINDER = B_PACK_CHARS('w', 'b', '*', B_TYPE_LARGE),  
    BINDER_TYPE_HANDLE  = B_PACK_CHARS('s', 'h', '*', B_TYPE_LARGE),  
    BINDER_TYPE_WEAK_HANDLE = B_PACK_CHARS('w', 'h', '*', B_TYPE_LARGE),  
    BINDER_TYPE_FD      = B_PACK_CHARS('f', 'd', '*', B_TYPE_LARGE),  
};

在binder里关键的结构体flat_binder_object中，主要看的type是BINDER_TYPE_FD，要传输的文件描述符的值保存在handle域中。

文件描述符类型的Binder对象在Binder驱动程序中的相关处理逻辑实现在binder_transact函数。

binder_transact函数核心功能：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
case BINDER_TYPE_FD: {  
    int target_fd;  
    struct file *file;  
    if (reply) {  
        if (!(in_reply_to->flags & TF_ACCEPT_FDS)) { 
            return_error = BR_FAILED_REPLY;  
            goto err_fd_not_allowed;  
        }  
    } else if (!target_node->accept_fds) {  
        return_error = BR_FAILED_REPLY;  
        goto err_fd_not_allowed;  
    }  
    file = fget(fp->handle);  
    if (file == NULL) {  
        return_error = BR_FAILED_REPLY;  
        goto err_fget_failed;  
    }  
    target_fd = task_get_unused_fd_flags(target_proc, O_CLOEXEC);  
    if (target_fd < 0) {  
        fput(file);  
        return_error = BR_FAILED_REPLY;  
        goto err_get_unused_fd_failed;  
    }  
    task_fd_install(target_proc, target_fd, file);  
     
    /* TODO: fput? */  
    fp->handle = target_fd;  
} break;

代码地址：kernel/common/drivers/staging/android/binder.c
最后的功能非常复杂，以后会在详细解析上来分析。