Recently learned knowledge ABTest

Foreword

Only the head can become strong.

Text has been included to my GitHub repository, welcome Star: github.com/ZhongFuChen...

If you read my previous article, the students will know that I do at work is a push system, wrote before the band you understand what is pushing the Push message , which also mentioned that we might be able to do ABTest , and ultimately improve the click of a push message rate.

What is ABTest it? This article with your entry about.

A, ABTest introduction

For example, I wrote an article about ABTest, I want to read this article amount of energy on 2500, but I thought a good headline what is more appropriate. Next Tweet title very can affect the amount of reading, so I think a few titles:

I recently learned knowledge AbTest
AbTest entry

And I do not know what the effect will be better title, so I was having such a try:

"I recently learned AbTest knowledge," this title pushed to 10% of users
"AbTest Getting Started" this title pushed to 10% of users
After some time, I look at the results, the higher the amount of reading the title of which, I will be pushed to a higher effect title remaining 80% of users

Note that: When push the article, in addition to a different title, other factors need the same (not be to interfere with other factors), so look at the data when only convincing.

1.1 Why do ABTest?

ABTest The reason is very simple, we are doing business there will be a wide variety of ideas, such as:

"I think adding emoji expression on the copy, the click-through rate pushed message certainly higher."
"I think this button / replaced by another color pictures, the conversion rate will definitely increase."
"I think the home should be designed so that there should be such a wall chart so .."
.....

However, and not all of the ideas are correct , it is probably because you think to get rid of home styles, users do not like, it affected GMVso on and so on ....

A good product is iterative out, and we probably do not know this iteration ultimately good or bad (at least we think is good for users iterations, is helpful, the transformation of the company is good) , but our customers will not buy it.

Thus, in order to reduce the cost of trial and error , we do ABTest. A function to do it, we just put a little traffic facie effect, if the effect is worse than the original function, it is likely the idea that we did not meet expectations. If a small flow is better than expected, then gradually increase the flow until the full amount.

Second, how do ABTest?

From the above case, in fact, we probably know, ABTest most important is to do a shunt thing

The 10% flow points to the user groups A
The 10% flow points to the user groups A

We need to ensure that: a user request comes again, the result is the same as the user sees

For example, I visited Java3y, his profile is: "a public Java technology adhere to the original number." And an hour later, I again visited him once, his profile is: "a full technical dry numbers." And an hour later, I visited him once, his profile is: "a public Java technology adhere to the original number."

It is unreasonable, should the content on the user over a period of time to see is the same, or give users a sense of confusion.

OK, so in general you can do:

A user ID (device ID / CookieId / userId / openId) take the hash value Hash each result is the same.
Direct access to a bit in the user ID

Now look, ABTest if that is a diversion of stuff, but still took a big name is ABTest.

2.1 ABTest more content

If I did ABTest on a UI level, occupying 80% of the station's traffic, now I want to do a search result ABTest how to do? Only with the remaining 20% of? That my traffic is not enough ah (I could do all kinds of experiments it). ABTest and ABTest search results on the UI level can not simultaneously ah?

The answer is yes. Because the UI level and search results (Algorithms) of the business association is very low of. If you do " with a flow rate at the same time and do a search result ABTest on the UI level" to ensure that " at the UI level ABTest do not affect the search results ABTest "

The most widely used industry, is superimposable tub delaminating method
Flow between the layers without disturbing each other, which is talked about a lot of articles orthogonal ( traffic is re-broken in each layer will )

Source: https: //www.infoq.cn/article/BuP18dsaPyAg-hflDxPf

We can be so dry:% by 1000 to achieve a similar approach Hash (userId, LayerId)

Each layer regardless of the number of experiments, the impact on other layers are uniform

I understand:

To achieve nothing associated with the UI / algorithms / advertising these services, can use the same traffic do ABTest test, so divided layer. Flow through each layer needs to be broken up again flow (orthogonal) ---- After each experiment, the lower layer does not affect the experiment

If a strong business association should be on the same level, with more than one experiment are mutually exclusive (for example, a button color changed to green as an experiment, a button to change the style of the thumb as an experiment. Flow of these two experiments are to be mutually exclusive (or else do you know the user is because your button and click on the color or style)

At last

ABTest a complete system, not just only diversion, but also provides a convenient back-office systems can be configured to the user (our programmers), done experiments provide data reports show, etc., etc. -

Public micro-channel number does not support outside the chain, reply "in the background AB " to get more information ABTest

References:

Willing output of dry cargo of Java technology public number: Java3y . A public number more than 200 original articles technical articles, massive video resources, beautiful mind map, attention can get!

Forwarded to the circle of friends is the biggest support for me!

I think the article is well written, points praise !

Recent Recommended: lowest buy cloud server to build tutorials +