The Solution of Live Streaming Quiz Mobile Client

Live Streaming Quiz(LSQ) is a very prevalent interact game from Q4 2017. Many live streaming video companies all published own quiz feature in Appstore. Today I introduce my project of LSQ.

1. The Way of Playing Quiz

Live Streaming quiz is like joining “Slumdog Millionaire” in your smart phone. There are 10 questions asked by program anchor during the game. The topics of questions are about culture, language, history, music, movie stars and etc. The player have to elect one answer from 4 options in 10 seconds. If you are lucky, select correct all questions, you will divide equally one million dollar with all winner; if else the first six questions are selected right, you will also get some coupons of in-app purchase. When you lose the game, you will become a normal live video audience. But if you have Resurrect Card, you will return back the game.

HQ

Is it an exciting game? Let me talk about how to implement a LSQ mobile client framework.

2. Technical Workflow

The technical essence of live streaming quiz is combine of live video streaming, instance message and big data.

  • Live Video Streaming

    I have told about technology of live streaming video in the past post. Besides, in LSQ we used an interesting technology for video streaming called Supplemental Enhancement Information (SEI). SEI is used for synchronization between video content and socket message. It makes streaming to transmit more information (e.g. time stamp, json structure) excluding video. You can learn more in google patent image about SEI

  • Instance Message

    LSQ depends keep alive socket to implement many features, such as instance recieving question content, question result, user’s comment and online user number. Because there is time offset between the time of client recieving question send by Technical Director and Program Anchor speaking video, video streaming is added SEI time stamp. The delay time is approximately 4s ~ 15s. When the client recieving question content, client analyze the time stamp and save question in memory. When time stamp of streaming SEI greater than the time stamp from instance message, client will display the question panel.

  • Big Data

    When users make their choice, the server asynchronic the different answers. Big data has to advanced perfermance compute the answer and correct of question and usage rate of Resurrect Card . It also need to product a report form in data dashboard for technical director and other supervisors.

3. Mobile Client Life Cycle

The left arrow means posting data or displaying panel from client; and the right arrow means client receiveing data.

This is a round game’s life cycle in client. Client program needs to instancely response based on different information from SEI and socket.

4. Mobile Client Structure Graph

The core of LSQ client structure is data management. The data manager collects kinds type of data from IM, SEI and HTTP request. And data manager is unique data communication party exclude video, that is in order to decouple UI and data.

YXQueue

An OOP and easily using job queue for iOS

YXQueue is encapsulate for NSOperation. Thread’s manager and invoker are divided by YXQueue. Using it, developers won’t focus too mach on thread management, just pay attention to how to create a job and implement delegate.

The github address is https://github.com/jacklandrin/YXQueue

Architecture

  • YXQueueDispatcher

    It’s designed as dispatcher of all YXQueues. It maintains the NSOperationQueue for all jobs.

  • YXQueueJob

    You can understand job as a model for operations. Configration of operations is set here.

  • YXQueueJobManager

    It manages operations producted by job. Cause dependencies of operation, maybe YXQueueJobManager needs to manage multioperation for one job.

  • YXQueueOperation

    It inherits from NSOperation. You can implement your operation content in - (void)executeTaskWithResultBlock:(void (^)(void))block

  • <YXQueueJobDelegate>

    It provides job’s callback of finishing, starting, canceling and progress changing.

Usage

1. Inheriting YXQueueEngine

YXQueue provides YXDownloadQueue to multithread download big file. It would be seen as a demo for thread’s manager.

Firstly, implementing a subclass for YXQueueJob. Adding necessary properties of model, and configing job’s type, appropriate class of YXQueueJobManager and YXQueueOperation. Such as YXQueueDownloadJob:

@interface YXQueueDownloadJob : YXQueueJob

@property (nonatomic, strong) NSString *downloadUrl;
@property (nonatomic, strong) NSString *targePath;

@end

config:

- (NSString *)jobTypeString
{
    return @"download";
}

+ (Class)managerClass
{
    return [YXQueueDownloadJobManager class];
}

+ (Class)operationClass
{
    return [YXQueueDownloadOperation class];
}

Subsenquence, create YXQueueDownloadOperation inheriting from YXQueueOperation. Config operationModel, resourceIdentifier(thread’s name) and appropriate class of job. YXQueueOperationModel can rule the max concurrent thread count and operation type. Implement the method - (void)executeTaskWithResultBlock:(void (^)(void))block.

- (instancetype)initWithJob:(YXQueueJob *)queueJob
{
    NSAssert([queueJob isKindOfClass:[YXQueueDownloadJob class]], @"queueJob must be YXQueueDownloadJob");
    if (self = [super initWithJob:queueJob]) {
        self.resourceIdentifier = @"com.queue.download";
        self.queuePriority = NSOperationQueuePriorityLow;
        _session = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration] delegate:self delegateQueue:[NSOperationQueue mainQueue]];
    }
    return self;
}

- (YXQueueDownloadJob *)job
{
    return (YXQueueDownloadJob*)_job;
}

- (YXQueueOperationModel *)operationModel
{
    if (!_model) {
        _model = [[YXQueueOperationModel alloc] init];
        _model.operationTypeString = @"downloadOperation";
        _model.maxConcurrentOperationCount = 5;
    }
    return _model;
}

- (void)executeTaskWithResultBlock:(void (^)(void))block
{
    __weak typeof(self) weakSelf = self;
    NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:self.job.downloadUrl] cachePolicy:NSURLRequestReloadIgnoringLocalCacheData timeoutInterval:3600];
    NSURLSessionDownloadTask *downloadTask = [self downloadTaskWithRequest:request progress:^(NSProgress *downloadProgress) {
        weakSelf.progress = (float)downloadProgress.completedUnitCount / (float)downloadProgress.totalUnitCount;
        [weakSelf notifiProgressDidChange];
    } destination:^NSURL *(NSURL *targetPath, NSURLResponse *response) {
        return [NSURL fileURLWithPath:weakSelf.job.targePath];
    } success:^(NSURLResponse *response, NSURL *fileURL) {
        self.operationReslut = response;
        if (block) {
            block();
        }
    } failure:^(NSURLResponse *response, NSError *error) {
        weakSelf.operationError = error;
        if (block) {
            block();
        }
    }];
    downloadTask.priority = NSOperationQueuePriorityLow;
}

Finally, you can inherit a subclass YXQueueDownJobManager from YXQueueJobManager, though there isn’t any difference with superclass.

2. Creating a Job

You can create a job like this:

YXQueueDownloadJob *job = [[YXQueueDownloadJob alloc] init];
job.downloadUrl = @"https://www.exmaple.mp4";
job.targePath = targetUrl;
[job addDelegate:self];

//command the job to start.
[job doJob];

//command a non-current job to cancel.
[job doCancel];

and you can register these delegate methods to recieve change of job’s status:

/**
 job finished
 */
- (void)queueJob:(YXQueueJob*)job operationFinished:(YXQueueOperation*)operation;
/**
 job started
 */
- (void)queueJob:(YXQueueJob*)job operationDidStart:(YXQueueOperation*)operation;
/**
 job failed
 */
- (void)queueJob:(YXQueueJob*)job operationFailed:(YXQueueOperation*)operation withError:(NSError*)error;
/**
 job was cancelled
 */
- (void)queueJob:(YXQueueJob*)job operationDidCanceled:(YXQueueOperation*)operation;
/**
 the progress updated
 */
- (void)queueJob:(YXQueueJob*)job operationDidUpdateProgress:(float)progress;

Demo

YXQueueDemo is a mp4 downloader as a YXQueue’s demo provided for you. You can modify the mp4 URL to download different video, and the default max concorrent download count is 5, it’s set in YXQueueDownloadOperation‘s method operationModel.

YXStackView

UIToolbar is an useful control in UIKit. But after iOS 11 we need to add some compatible code to keep UI layout. Then I found UIStackView can be used as layout. However it doesn’t have some function like UIBarButtonSystemItemFlexibleSpace. So in order to create a container view supporting flexible space, I wrote YXStackView.

The github address is https://github.com/jacklandrin/YXStackView.

typedef enum {
    YXStackViewItemStyleCustom,
    YXStackViewItemStyleFlexibleSpace,
    YXStackViewItemStyleFlexibleItem
}YXStackViewItemStyle;

@interface YXStackViewItem : NSObject

@property (nonatomic, strong) UIView *customView;
@property (nonatomic, assign) NSUInteger index;
@property (nonatomic, assign) YXStackViewItemStyle style;

@end

The class YXStackViewItem is model of item view in container, it likes UIBarButtonItem in UIToolbar. If the YXStackViewItemStyle is YXStackViewItemStyleFlexibleSpace, the customView will be nil. You can instantiate items based your requirement and set them into the YXStackView.

typedef enum {
    YXStackViewAxisHorizontal,
    YXStackViewAxisVertical
} YXStackViewAxis;

@interface YXStackView : UIView

@property (nonatomic, assign) YXStackViewAxis axis;
@property (nonatomic, assign) CGFloat spacing;
@property (nonatomic, assign) CGFloat columnSpacing;//When isAutoFitEdge is YES, it's avilable
@property (nonatomic, assign) BOOL reverse; //When isAutoFitEdge is NO, it's avilable
@property (nonatomic, strong) NSArray<YXStackViewItem*>* items;
@property (nonatomic, assign) BOOL isAutoFitEdge; //whether item is auto resizing with View,if over view edge stackView is multiline. If it's YES,YXStackViewItem doesn't support YXStackViewItemStyleFlexibleSpace and YXStackViewItemStyleFlexibleItem.

@end

The YXStackView supports horizontal & vertical two layout orientations, and whether reverse sorted items. isAutoFitEdge can control if multiline display, if it’s YES, the columnSpacing will be avilable. When layoutSubView is invoked, the items’ layout will be recoculated.

If you just use it to instead of UIToolbar, you can write like this:

YXStackView *toolbar = [[YXStackView alloc] initWithFrame:CGRectMake(0, 200, self.view.frame.size.width, 40)];
toolbar.spacing = 10.0;
toolbar.isAutoFitEdge = NO;


[self.view addSubview:toolbar];

UIView *view1 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 200, 30)];
view1.backgroundColor = [UIColor redColor];

YXStackViewItem *item1 = [[YXStackViewItem alloc] init];
item1.style = YXStackViewItemStyleCustom;
item1.customView = view1;


YXStackViewItem *space = [[YXStackViewItem alloc] init];
space.style = YXStackViewItemStyleFlexibleSpace;


UIView *view3 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 100, 40)];
view3.backgroundColor = [UIColor purpleColor];

YXStackViewItem *item3 = [[YXStackViewItem alloc] init];
item3.style = YXStackViewItemStyleCustom;
item3.customView = view3;


[toolbar setItems:@[item1,space,item3]];

and you will get this layout:

If else UIStackView mode:

YXStackView *toolbar = [[YXStackView alloc] initWithFrame:CGRectMake(0, 200, self.view.frame.size.width, 40)];
    toolbar.spacing = 10.0;
    toolbar.isAutoFitEdge = YES;

    [self.view addSubview:toolbar];

    UIView *view1 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 200, 30)];
    view1.backgroundColor = [UIColor redColor];

    YXStackViewItem *item1 = [[YXStackViewItem alloc] init];
    item1.style = YXStackViewItemStyleCustom;
    item1.customView = view1;

    UIView *view2 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 250, 30)];
    view2.backgroundColor = [UIColor blueColor];

    YXStackViewItem *item2 = [[YXStackViewItem alloc] init];
    item2.style = YXStackViewItemStyleCustom;
    item2.customView = view2;

    UIView *view3 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 100, 40)];
    view3.backgroundColor = [UIColor purpleColor];

    YXStackViewItem *item3 = [[YXStackViewItem alloc] init];
    item3.style = YXStackViewItemStyleCustom;
    item3.customView = view3;

    UIView *view4 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 30, 30)];
    view4.backgroundColor = [UIColor greenColor];

    YXStackViewItem *item4 = [[YXStackViewItem alloc] init];
    item4.style = YXStackViewItemStyleCustom;
    item4.customView = view4;

    UIView *view5 = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 20, 30)];
    view5.backgroundColor = [UIColor brownColor];

    YXStackViewItem *item5 = [[YXStackViewItem alloc] init];
    item5.style = YXStackViewItemStyleCustom;
    item5.customView = view5;

    toolbar.columnSpacing = 10;
    [toolbar setItems:@[item1, item2, item3, item4, item5]];

and you will got :

An Amazing Class: NSProxy

NSProxy is a root class in Objective-C. yes, NSObject is not unique root class. From the definition of NSProxy we can think NSProxy as a simplified NSObject. It just implements protocol. As an abstract class, the methods need to be implemented by subclass. One of them forwardInvocation: is the most key method of this class, and it can implement a part of feature of fowarding message.

Typically, proxy is used to implement delegate pattern. For example, making an animal proxy:

//AnimalProxy.h

@interface AnimalProxy : NSProxy

- (void)proxyWithAnimal:(NSObject*)anObject;

@end

@interface Bird : NSObject

- (void)fly;

@end

@interface Tiger : NSObject

- (void)eat:(NSString*)food;

@end

//AnimalProxy.m

@interface AnimalProxy()

@property (nonatomic, strong) NSObject *proxyObject;

@end

@implementation AnimalProxy

- (void)proxyWithAnimal:(NSObject *)anObject
{
    self.proxyObject = anObject;
}

- (void)forwardInvocation:(NSInvocation *)invocation
{
    if (self.proxyObject) {

        [invocation setTarget:self.proxyObject];

        if ([self.proxyObject isKindOfClass:[NSClassFromString(@"Tiger") class]]) {
            NSString *str = @"deer";
            [invocation setArgument:&str atIndex:2];
        }

        [invocation invoke];
    }
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel
{
    NSMethodSignature *signature = nil;
    if ([self.proxyObject methodSignatureForSelector:sel]) {
        signature = [self.proxyObject methodSignatureForSelector:sel];
    } else {
        signature = [super methodSignatureForSelector:sel];
    }

    return signature;
}

@end

@implementation Bird

- (void)fly
{
    NSLog(@"Bird flies");
}

@end

@implementation Tiger

- (void)eat:(NSString *)food
{
    NSLog(@"Tiger eats %@",food);
}

@end

When I invoked them, I counld get output like this:

//NSProxy doesn't have initialization method
AnimalProxy *proxy = [AnimalProxy alloc];     
Tiger *tiger = [[Tiger alloc] init]; 
Bird *bird = [[Bird alloc] init];

[proxy proxyWithAnimal:tiger];  
[proxy performSelector:@selector(eat:) withObject:@"zebra"];

[proxy proxyWithAnimal:bird];   
[proxy performSelector:@selector(fly)];

output:
2018-05-21 21:30:26.866892+0800 MethodDemo[3860:852618] Tiger eats deer
2018-05-21 21:30:26.867248+0800 MethodDemo[3860:852618] Bird flies

Developers can use NSProxy to finish many function, such as decoupling, AOP, method interception and etc. A smart gay called ibireme wrote a proxy to solve that NSTimer can’t dealloc with CADisplayLink. Let’s see his code from github.

@interface YYWeakProxy : NSProxy

/**
 The proxy target.
 */
@property (nullable, nonatomic, weak, readonly) id target;

/**
 Creates a new weak proxy for target.

 @param target Target object.

 @return A new proxy object.
 */
- (instancetype)initWithTarget:(id)target;

/**
 Creates a new weak proxy for target.

 @param target Target object.

 @return A new proxy object.
 */
+ (instancetype)proxyWithTarget:(id)target;

@end
@implementation YYWeakProxy

- (instancetype)initWithTarget:(id)target {
    _target = target;
    return self;
}

+ (instancetype)proxyWithTarget:(id)target {
    return [[YYWeakProxy alloc] initWithTarget:target];
}

- (id)forwardingTargetForSelector:(SEL)selector {
    return _target;
}

- (void)forwardInvocation:(NSInvocation *)invocation {
    void *null = NULL;
    [invocation setReturnValue:&null];
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)selector {
    return [NSObject instanceMethodSignatureForSelector:@selector(init)];
}

- (BOOL)respondsToSelector:(SEL)aSelector {
    return [_target respondsToSelector:aSelector];
}

- (BOOL)isEqual:(id)object {
    return [_target isEqual:object];
}

- (NSUInteger)hash {
    return [_target hash];
}

- (Class)superclass {
    return [_target superclass];
}

- (Class)class {
    return [_target class];
}

- (BOOL)isKindOfClass:(Class)aClass {
    return [_target isKindOfClass:aClass];
}

- (BOOL)isMemberOfClass:(Class)aClass {
    return [_target isMemberOfClass:aClass];
}

- (BOOL)conformsToProtocol:(Protocol *)aProtocol {
    return [_target conformsToProtocol:aProtocol];
}

- (BOOL)isProxy {
    return YES;
}

- (NSString *)description {
    return [_target description];
}

- (NSString *)debugDescription {
    return [_target debugDescription];
}

Method in Objective-C : Message Passing

Method in Objective-C : Message Passing

What is Message Passing?

At previous post I mentioned Method Swizzling, that refers some knowledge about message passing. So this post I decide to talk about it.

We know invoking an instance method called Message Passing in Objective-C. For example:

NSMutableArray *array = [NSMutableArray array];
[array addObject:@"hello world!"];

array is message receiver, addObject calls as selector that I mentioned at last post. message consists of selector and parameter.

Message passing adopt dynamic binding to decide to invoke which method. Dynamic binding means that the compiler doesn’t know which method implementation will be selected; instead the method implementation is looked up at runtime when the message is sent. So we write some code:

id num = @123;
//output 123
NSLog(@"%@", num);
//crash, error: [__NSCFNumber appendString:]: unrecognized selector sent to instance 0x8c28
[num appendString:@"Hello World"];

During compiling above code doesn’t have any problems, because id type can point all kinds of instance. NSString has a method appendString: ; compiler doesn’t sure which type instance num is, and add a new method to NSNumber is available in project running, thus it wouldn’t show error when compiler found appendString: method declaration. But in project running can’t find appendString: in NSNumber, program would be error. So this is a disadvantage of message passing, compiler can’t check method undefined.

You can learn more information about Message at Apple development website.

objc_msgSend()

Runtime provides a C language method for sending message, it’s objc_msgSend(receiver, selector, arg1, arg2, …). The first parameter is receiver that is method’s invoker, and second parameter is selector that is like method’s name, there are several arguments behind it. If you want to get C language function from your Objective-C method, there is a cool command:

clang -rewrite-objc main.m

this command can help you convert objc file extended .m to .cpp file.

So we have this code:

@interface Macbook : NSObject

@property (nonatomic, copy) NSString *type;
@property (nonatomic, assign) CGFloat price;

- (void)showPriceTag;

@end

@implementation Macbook

- (void)showPriceTag
{
    NSLog(@"This laptop type is %@, and price is $%.2f",self.type, self.price);
}

@end

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        Macbook *macbookpro = [[Macbook alloc] init];
        macbookpro.type = @"Macbook Pro Retina";
        macbookpro.price = 199.9;
        [macbookpro showPriceTag];
    }
    return 0;
}

When I used clang command to convert code, I got a file called main.cpp in same folder, and I found these definitions:

#ifndef _REWRITER_typedef_Macbook
#define _REWRITER_typedef_Macbook
typedef struct objc_object Macbook;
typedef struct {} _objc_exc_Macbook;
#endif

extern "C" unsigned long OBJC_IVAR_$_Macbook$_type;
extern "C" unsigned long OBJC_IVAR_$_Macbook$_price;
struct Macbook_IMPL {
    struct NSObject_IMPL NSObject_IVARS;
    NSString *_type;
    CGFloat _price;
};


// @property (nonatomic, copy) NSString *type;
// @property (nonatomic, assign) CGFloat price;

// - (void)showPriceTag;

/* @end */


// @implementation Macbook


static void _I_Macbook_showPriceTag(Macbook * self, SEL _cmd) {
    NSLog((NSString *)&__NSConstantStringImpl__var_folders_9t_z6zhv5ys04q0bby1zkgnm14h0000gn_T_main_a83434_mi_0,((NSString *(*)(id, SEL))(void *)objc_msgSend)((id)self, sel_registerName("type")), ((CGFloat (*)(id, SEL))(void *)objc_msgSend_fpret)((id)self, sel_registerName("price")));
}


static NSString * _I_Macbook_type(Macbook * self, SEL _cmd) { return (*(NSString **)((char *)self + OBJC_IVAR_$_Macbook$_type)); }
extern "C" __declspec(dllimport) void objc_setProperty (id, SEL, long, id, bool, bool);

static void _I_Macbook_setType_(Macbook * self, SEL _cmd, NSString *type) { objc_setProperty (self, _cmd, __OFFSETOFIVAR__(struct Macbook, _type), (id)type, 0, 1); }

static CGFloat _I_Macbook_price(Macbook * self, SEL _cmd) { return (*(CGFloat *)((char *)self + OBJC_IVAR_$_Macbook$_price)); }
static void _I_Macbook_setPrice_(Macbook * self, SEL _cmd, CGFloat price) { (*(CGFloat *)((char *)self + OBJC_IVAR_$_Macbook$_price)) = price; }
// @end

int main(int argc, const char * argv[]) {
    /* @autoreleasepool */ { __AtAutoreleasePool __autoreleasepool; 
        Macbook *macbookpro = ((Macbook *(*)(id, SEL))(void *)objc_msgSend)((id)((Macbook *(*)(id, SEL))(void *)objc_msgSend)((id)objc_getClass("Macbook"), sel_registerName("alloc")), sel_registerName("init"));
        ((void (*)(id, SEL, NSString *))(void *)objc_msgSend)((id)macbookpro, sel_registerName("setType:"), (NSString *)&__NSConstantStringImpl__var_folders_9t_z6zhv5ys04q0bby1zkgnm14h0000gn_T_main_a83434_mi_1);
        ((void (*)(id, SEL, CGFloat))(void *)objc_msgSend)((id)macbookpro, sel_registerName("setPrice:"), 199.90000000000001);
        ((void (*)(id, SEL))(void *)objc_msgSend)((id)macbookpro, sel_registerName("showPriceTag"));
    }
    return 0;
}

The properties generate getter and setter method, so they are methods as well. We can see objc_msgSend like this:

((void (*)(id, SEL, CGFloat))(void *)objc_msgSend)((id)macbookpro, sel_registerName("setPrice:"), 199.90000000000001);
  • 1.register method name
  • 2.send message with an argument 199.90000000000001 to reciever macbookpro by objc_msgSend

You should learn runtime how to implement a invoking of Object-Oriented to pass message in objc_msgSend. objc_msgSend can select appropriate method to invoke according to reciever. How does it work? We need to learn structure of Class in Runtime.

struct objc_class {
    Class isa  OBJC_ISA_AVAILABILITY;

    Class super_class                                        
    const char *name                                         
    long version                                             
    long info                                                
    long instance_size                                       
    struct objc_ivar_list *ivars                             
    struct objc_method_list **methodLists                    
    struct objc_cache *cache                                 
    struct objc_protocol_list *protocols                     
}
/* Use `Class` instead of `struct objc_class *` */

/// An opaque type that represents an Objective-C class.
typedef struct objc_class *Class;

/// Represents an instance of a class.
struct objc_object {
    Class isa  OBJC_ISA_AVAILABILITY;
};

/// A pointer to an instance of a class.
typedef struct objc_object *id;

You can look for Runtime open source code in here.

The struct objc_class* contains a member variable struct objc_method_list **methodLists, and then found out definition of struct objc_method_list:

static struct /*_method_list_t*/ {
    unsigned int entsize;  // sizeof(struct _objc_method)
    unsigned int method_count;
    struct _objc_method method_list[5];
} _OBJC_$_INSTANCE_METHODS_Macbook __attribute__ ((used, section ("__DATA,__objc_const"))) = {
    sizeof(_objc_method),
    5,
    {{(struct objc_selector *)"showPriceTag", "v16@0:8", (void *)_I_Macbook_showPriceTag},
    {(struct objc_selector *)"type", "@16@0:8", (void *)_I_Macbook_type},
    {(struct objc_selector *)"setType:", "v24@0:8@16", (void *)_I_Macbook_setType_},
    {(struct objc_selector *)"price", "d16@0:8", (void *)_I_Macbook_price},
    {(struct objc_selector *)"setPrice:", "v24@0:8d16", (void *)_I_Macbook_setPrice_}}
};

struct _objc_method {
    struct objc_selector * _cmd;
    const char *method_type;
    void  *_imp;
};

The struct objc_method_list contains three memeber variable: method size, method count and the most important method list. Element of the method list is struct _objc_method, the struct consists of selector, method type and implementation pointer.

That can explain how does objc_msgSend work, runtime searches this struct objc_method_list in class of receiver, so that it can mach message’s reciever and selector. If method is found out, the _imp in struct _objc_method will be called; else runtime will go to super_class pointer by inheritance tree, find out method, jump. If the method is not find out, when reach the root of inheritance tree (normally is NSObject), a method of NSObject called doesNotRecognizeSelector: will be invoked. That is the error unrecognized selector which you often come across.

Message Forwarding

Actually there are three chances process message before doesNotRecognizeSelector: invoked. That is message forwarding. NSObject provides four method to implement message forwarding:

// 1
+(BOOL)resolveInstanceMethod:(SEL)sel{
    // A chance to add the instance method and return YES. It will then try sending the message again.
}

// 2
- (id)forwardingTargetForSelector:(SEL)aSelector{
    // Return an object that can handle the selector.
}

// 3
- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector{
    // You need to implement this for the creation of an NSInvocation.
}

- (void)forwardInvocation:(NSInvocation *)invocation {
    // Invoke the selector on a target of your choice.
    [invocation invokeWithTarget:target];
}

This is the flow of message forwarding:

  • First of all, program will send a request to reciever’s class, if it don’t find out relevant method by inheritance tree. +resolveInstanceMethod: tries to add a method dynamically.

  • Then Runtime will ask the reciever whether there is another instance can process this unknow selector in -forwardingTargetForSelector:, when current instance can’t add method. If it can return a backup reciever, the message will be processed, else if return nil, program will go to the next step.

  • The thrid chance is forwarding a invocation. At this time, the detail of all unknow message will encapsulate as a NSInvocation object, and invoke -forwardInvocation: method. This is the last chance, if NSObject can’t process the message, doesNotRecognizeSelector: will throw exception.

You can forward a method to another target. That’s very useful if you’re trying to bridge between different frameworks. This is what happens when you call a method that’s not implemented.

You can learn more information about Message Forwarding in Apple documents

Talking about Aspect Oriented Programming in iOS

1. What’s the Aspect Oriented Programming?

The Apsect Oriented Programming (AOP) is usually used in backend development. But it’s more and more popular in clients programming recent days.

You can find out the definition in Wikipedia:

In computing, AOP is a programming paradigm that aims to increase modularity by allowing the separation of cross-cutting concerns. It does so by adding additional behavior to existing code (an advice) without modifying the code itself, instead separately specifying which code is modified via a “pointcut” specification, such as “log all function calls when the function’s name begins with ‘set'”. This allows behaviors that are not central to the business logic (such as logging) to be added to a program without cluttering the code, core to the functionality. AOP forms a basis for aspect-oriented software development.

2. How to implement AOP in iOS?

When we develop a statistic system of client, sometimes we don’t want add any code to the business logic. AOP is a good choice for some situation of logging. We can hook methods in which we need to make some logs. Such as logging page appear and disappear.

Method Swizzling

In iOS programming, we can implement AOP by Method Swizzling. Method Swizzling is way that replace original method with another.

Talking about some Runtime knowledge is necessary to how to explain Method Swizzling. Method in Objective-C consists of SEL(Selector) and IMP(implemention).

  • Method selectors are used to represent the name of a method at runtime. A method selector is a C string that has been registered (or “mapped“) with the Objective-C runtime. Selectors generated by the compiler are automatically mapped by the runtime when the class is loaded.

  • IMP is a pointer to the start of the function that implements the method.

Method Swizzling exchanges IMPs mapped selectors in order to replace methods. Implementing a Method Swizzling, the core C language API at runtime is:

OBJC_EXPORT void method_exchangeImplementations(Method m1, Method m2) 
 __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);

Method Swizzling Example

Suppose we want to print out a log statement whenever -viewDidLoad() is called on a UIViewController. Creating a category of UIViewController, and implement its +load().

#import "UIViewController+swizzling.h"
#import @implementation UIViewController (swizzling)

+ (void)load {
    [super load];
    // class_getInstanceMethod() get method structure from method list
    Method fromMethod = class_getInstanceMethod([self class], @selector(viewDidLoad));
    Method toMethod = class_getInstanceMethod([self class], @selector(swizzling_ViewDidLoad));

    if (!class_addMethod([self class], @selector(viewDidLoad), method_getImplementation(toMethod), method_getTypeEncoding(toMethod))) {
        method_exchangeImplementations(fromMethod, toMethod);
    }
}

//  define method by myself, replacing original method
- (void)swizzling_ViewDidLoad {
    NSLog(@"Log something... for %@",[self class]);
    [self swizzling_ViewDidLoad];
}
@end

3. Aspects

Obviously, if we use this way to hook every method needed to add log, it will be a huge project. Maybe there is a third party libarary that we use easier. YES, Aspects is an awesome tool for AOP in Objective-C.

Aspects provides two methods for developer:

+ (id<AspectToken>)aspect_hookSelector:(SEL)selector
                      withOptions:(AspectOptions)options
                       usingBlock:(id)block
                            error:(NSError **)error;
- (id<AspectToken>)aspect_hookSelector:(SEL)selector
                      withOptions:(AspectOptions)options
                       usingBlock:(id)block
                            error:(NSError **)error;

An instance method and a class method, they have same name and same parameters. It’s thread safe for all invoking, and Aspects makes some perfermance loss due to message passing of Objective-C. Therefore I don’t suggest use Aspects for methods in frequent invoked.

Example

I use the Aspects instead of complex Method Swizzling, hook -viewWillApear() in all UIViewControllers.

#import "ViewControllerLogger.h"
#import @implementation ViewControllerLogger

- (instancetype)init {
    if (self = [super init]) {
        [UIViewController aspect_hookSelector:@selector(viewWillAppear:) withOptions:AspectPositionAfter usingBlock:^(id<AspectInfo> aspectInfo) {
            NSLog(@"ViewController will appear:--> %@", NSStringFromClass([aspectInfo.instance class]));
        } error:NULL];
    }
}
@end

Disadvantage

There are some disadvantages at Aspects.

  • Perfermance loss

    I have memtioned that above paragraph.

  • Can’t hook class method

    If you hook a class method, the block won’t callback and you’ll get failure said Aspects: Block signature doesn’t match (null). The reason is that runtime just obtain member method list, if you want to get class method, you must get it from MetaClass of this class. You can get MetaClass by object_getClass(newClass).

    Fortunately, there is a gay coding a function for class method hooking:

    static void aspect_prepareClassAndHookSelector(NSObject *self, SEL selector, NSError **error) {
    NSCParameterAssert(selector);
    Class klass = aspect_hookClass(self, error);
    //TODO:Edit by JackYong
    Method targetMethod;
    IMP targetMethodIMP;
    if (class_isMetaClass(klass)) {
        targetMethod = class_getClassMethod(klass, selector);
        targetMethodIMP = method_getImplementation(targetMethod);
    } else {
        targetMethod = class_getInstanceMethod(klass, selector);
        targetMethodIMP = method_getImplementation(targetMethod);
    }
    

How to improve live video streaming?

1. How live video streaming works?

There are these key parts in basic streaming process:

  • Caputure or media resource
  • Broadcaster
  • Source server
  • Recorder for playback
  • Content Delivery Network (CDN) nodes
  • Player Clients

The process of live video streaming: The broadcaster (smart phone, OBS or other device) puts streaming from caputure or media resource to source server in Real-Time Messaging Protocol (RTMP) .

The source server’s major duty is transcoding. If clients need playback, the source server could record video in real time and save data in cloud server.

And next, every CDN node can get the streaming from source server, if there are some player clients nearby the CDN. CDN has two functions: loading balance and cache. Then the streaming would be pulled from CDN to clients in suitable protocol.

Like these not complex steps, you can watch the vivid video on your phone.

2. What are the difference in kind of live video protocols?

When I was as a beginner of live video streaming developer, I fell in comfuse in these live streaming protocols. I worked hard to find the difference in them, and made a summary.

  • Http live streaming (HLS)

    HLS is an HTTP-based media streaming communications protocol implemented by Apple Inc. as part of its QuickTime, Safari, OS X, and iOS software. It resembles MPEG-DASH in that it works by breaking the overall stream into a sequence of small HTTP-based file downloads, each download loading one short chunk of an overall potentially unbounded transport stream. As the stream is played, the client may select from a number of different alternate streams containing the same material encoded at a variety of data rates, allowing the streaming session to adapt to the available data rate. At the start of the streaming session, HLS downloads an extended M3U playlist containing the metadata for the various sub-streams that are available. Wikipedia-EN

    The format of M3U8 playlist:

    #EXTM3U                     m3u header, must be written in first lin
    #EXT-X-MEDIA-SEQUENCE       sequence of the first .ts segment
    #EXT-X-TARGETDURATION       maximium time duration of each segment
    #EXT-X-ALLOW-CACHE          allow cache
    #EXT-X-ENDLIST              end signal of m3u8 file 
    #EXTINF                     extra info,info of .ts segment, such as duration, brandwidth, etc.
    
    

    The plist contains many file paths of .ts segment:

    Meanwhile, you can learn more introduction from apple.

    When clients pull streaming in HLS, they need to download the current playlist generated by server at first, then download .ts segment one by one. So as source server, after generating a playlist, clients have possiblity of pulling this streaming. This is the reason why HLS has high latency problem. In non-HMTL envirement, this protocol isn’t recommended to use.

  • Real-Time Messaging Protocol (RTMP)

    RTMP is a TCP-based data communications protocol, it is applied to media streaming widely. This protocol is real-time transport streaming, so its latency is lower than HLS. The congestion control of TCP can ensure the data integrity. In addition, Adoube provides this protocol, Adoube Flash can play video streaming in RTMP perfectly.The most broadcasters provide this protocal to pushing streaming as well.

  • Http-flv

    Http-flv transports streaming extented flv in HTTP. HTTP has a key in header called content-length, it rules the body length of request. If don’t add content-length in header, clients will recieve data all the time. The packet based http-flv may be smaller than RTMP, so it has advantage at saving data flow. Besides, the latency of http-flv almost equels to RTMP.

  • UDP

    Most video chat apps perfer using protocol based-UDP, because UDP doesn’t have congestion control. This can decrease latency time to subsecond. But the reason why many companies don’t use it is that they need to refactor the framework of server. So as video chat companies, they have good advantage. Currently due to requirment of extra low latency, many online-education companies use UDP to implement live video streaming products.

3. The problem of live video streaming latency

Different business requirments lead to different technical selector. If customers wanna watch a live hot girl show, 2~5s latency maybe OK, but as online-course, about 1s just can ensure normal teaching needs. There are these reason influncing latency of live video:

  • Network speed
    In the whole process of live video streaming, whatever network speed of from broadcaster to source server or from CDN to player clients, all influence experience of live video. So network speed is a important factor of latency.

  • Live video protocol

    Last section, I have discussed the specify of kind of protocols, so I don’t talk about it more.

  • Video encoding algorithm

    To a large extent, an effective strategy or a good algorithm can reduce latency. Next section I will introduce video essential and cosider how to make an effective encoding strategy.

4. Video essential

We know that a video consists of many pictures in fact, the pictures are fast play over 24 fps to cheat our eyes. That makes people feel the screen is dynamic. But if a 1080p video playing in 60 fps and isn’t compressed, it will be a huge stuff. I guess one of an intact frame image in this video is about several MB. Therefore, innovating a video encoding for compressing is necessary.

There is a not bad thinking that we just save difference of similiar images in video. That redundancy is deleted makes video smaller.

Let a part of frame hold intact image or least compressible, we call them I frames(Intra-coded frames). If we can seek video to I frame, we call them Key frames. From an I frame to next I frame is a group of pictures, or GOP structure. Those frames holding differece are called reference frames. Predicted picture(P frame) holds only the changes in the image from the previous frame. Bidirectional predicted picture(B frame) saves even more space by using differences between the current frame and both the preceding and following frames to specify its content.
Learn more in Wikipedia

From these principles, we know when player decodes a video, B frame need to reference the preceding and following frames. Though B frame makes video size smaller, decoding will spend more time, because decoder need to wait for whole GOP or frame referenced by B frame downloaded in order to decode this B frame in this GOP. If the GOP is long, the time for waiting is long.

5. How does the first frame load fast?

  • Caching GOP

    As decoder of player,comes across an I frame, the video just will be played. We can suppose that player need to wait 2 second to recieve an I frame, we maybe see the black screen 2s. So player must fast load first I frame for swiftly playing live video. Many companies often make a strategy that cache the current live GOP in CDN. When player clients pull streaming, directly get the last I frame from CDN, it can ensure player plays video immediately.

  • Smaller buffer

    The player downloads a bit of data to buffer. When the buffer is empty, the decoder waits for data. When the buffer is full, the decoder works and you would see the content on the screen. The buffer of live video is hundred millisecodn level. Reducing size of buffer could help player to render image on time. But the disvantage is that if network status is bad, buffer often will be empty, user will see the caching tips on screen. So I suggest just reducing size of the first buffer to make image swiftly to show.

  • Selecting nearest CDN

    Don’t select a far CDN to pulling streaming, it will spend more time on network route. Perhaps ignoring loading balance is sometimes a good choice for live video.

  • Using UDP

    Changing protocol based-TCP to UDP that doesn’t hava congestion control.

  • Using IP in URL

    DNS takes long time on network request. Don’t use domain in URL of video, local DNS will be an excellent project for it.