Design and implementation of intelligent reply system based on image answer algorithm (front-end Android+back-end Python)

Contents
1 Introduction 1
1.1 Research background and significance 1
1.2 Question answering system research review 2
1.2.1 Intelligent reply system software 2
1.2.2 Question answering system research status at home and abroad 3
1.2.3 Image question answering review 4
1.3 Main problems 6
1.4 Research content 6
1.5 Technical route 6
1.6 Paper organization structure 7
2 Relevant theoretical knowledge 9
2.1 Question answering system 9
2.1.1 General processing flow of question answering system 9
2.1.2 Basic questions of question answering system research 9
2.1.3 Main methods of question answering system 9
2.2 Intelligence Intelligent Reply System11
2.2.1 Retrieval-Based Intelligent Reply System11
2.2.2 Structure of Intelligent Reply System Based on Generated Dialogue12
2.3 Image Question Answering12
2.3.1 Image Question Answering Algorithm Framework13
2.3.2 Convolutional Neural Network13
2.3.3 Long Short-Term Memory Network 15
3 System Design 18
3.1 Overall Function Design 18
3.2 Image Question Answering Algorithm Framework 18
3.3 Dataset 19
3.4 Dataset Preprocessing 20
3.5 Keras Framework Construction 21 3.5.1
Introduction to Keras 21
3.5.2 Keras Platform Construction 21
3.5.3Keras example 22
4 system implementation 23
4.1 Image question answering algorithm implementation 23
4.1.1 Image feature extraction 23
4.1.2Embedding layer 24
4.1.3 Model establishment 25
4.2 Image question answering algorithm model application 27
4.3 Algorithm testing 29
4.4 Server Design and Implementation 30
4.4.1 Development Environment 30
4.4.2 Server Function Design 30
4.4.3 Web.py Introduction and Installation 30
4.4.4 Turing Robot API Introduction and Use 31
4.4.5 Server Function Implementation 32
4.5 Client Design 34
4.5.1 Development Environment 34
4.5.2 Client Function Design 34
4.5.3 Client Main Class Diagram 34
4.5.4 Use Case Diagram 35
4.5.4 Client Sequence Diagram 35
4.5.5 Interface Design 36
4.5.6 Key Function Realization 38
5 System Test 43
5.1 Interface Test 43
5.2 Picture Function Test 44
5.3 Common Words Sending Test 46 5.4
Chat Function Test 47
6 Conclusion 51
References 52
Acknowledgments 54
4.4 Server Design and Implementation
4.4.1 Development Environment
System: Ubuntu 16.04
Editor: Vim7.4
Development language: python2.7
4.4.2 Server function design
The server mainly undertakes the function of receiving messages and returning messages, and integrates the Turing robot interface and the image question-and-answer algorithm.

Figure 20 Server function diagram
4.5.1 Development environment
System: Ubuntu 16.04
Integrated development tool: Android Studio2.2
Target adaptation system: Android 5.1
Development language: Java
4.5.2 Client function design
The client form is an Android application. The interface of this application is borrowed from the chat interface of WeChat, which mainly realizes the functions of sending messages, receiving messages, storing messages, sending pictures, taking pictures, viewing albums, and sending common words.

Figure 23 client function diagram

#/usr/bin/env python
#coding=utf8
 
import httplib
import md5
import urllib
import random
import json

appid = '20170420000045283'
secretKey = 'LaYwFOmUBRvZ9BOwxPyO'


def transFromZhToEn(q):
    httpClient = None
    myurl = '/api/trans/vip/translate'
   
    fromLang = 'zh'
    toLang = 'en'
    salt = random.randint(32768, 65536)

    sign = appid+q+str(salt)+secretKey
    m1 = md5.new()
    m1.update(sign)
    sign = m1.hexdigest()
    myurl = myurl+'?appid='+appid+'&q='+urllib.quote(q)+'&from='+fromLang+'&to='+toLang+'&salt='+str(salt)+'&sign='+sign
    result=""
    try:
        httpClient = httplib.HTTPConnection('api.fanyi.baidu.com')
        httpClient.request('GET', myurl)
    
        #response是HTTPResponse对象
        response = httpClient.getresponse()
        stre = response.read()
        res = json.loads(stre)
        tres = res["trans_result"][0]
        #reseu = json.loads(tres)
        #print res["trans_result"]
        result =  tres['dst']
        #print response.read()
        #print stre
    except Exception, e:
        print e
    finally:
        if httpClient:
            httpClient.close()
    
    return result

def transFromEnToZh(q):
    httpClient = None
    myurl = '/api/trans/vip/translate'
   
    fromLang = 'en'
    toLang = 'zh'
    salt = random.randint(32768, 65536)

    sign = appid+q+str(salt)+secretKey
    m1 = md5.new()
    m1.update(sign)
    sign = m1.hexdigest()
    myurl = myurl+'?appid='+appid+'&q='+urllib.quote(q)+'&from='+fromLang+'&to='+toLang+'&salt='+str(salt)+'&sign='+sign
    result=""
    try:
        httpClient = httplib.HTTPConnection('api.fanyi.baidu.com')
        httpClient.request('GET', myurl)
    
        #response是HTTPResponse对象
        response = httpClient.getresponse()
        stre = response.read()
        res = json.loads(stre)
        tres = res["trans_result"][0]
        #reseu = json.loads(tres)
        #print res["trans_result"]
        result =  tres['dst']
        #print response.read()
        #print stre
    except Exception, e:
        print e
    finally:
        if httpClient:
            httpClient.close()
    
    return result
 
httpClient = None
myurl = '/api/trans/vip/translate'
q = '这个人穿的衣服是什么颜色的?'
fromLang = 'zh'
toLang = 'en'
salt = random.randint(32768, 65536)

sign = appid+q+str(salt)+secretKey
m1 = md5.new()
m1.update(sign)
sign = m1.hexdigest()
myurl = myurl+'?appid='+appid+'&q='+urllib.quote(q)+'&from='+fromLang+'&to='+toLang+'&salt='+str(salt)+'&sign='+sign
 
try:
    httpClient = httplib.HTTPConnection('api.fanyi.baidu.com')
    httpClient.request('GET', myurl)
 
    #response是HTTPResponse对象
    response = httpClient.getresponse()
    stre = response.read()
    res = json.loads(stre)
    tres = res["trans_result"][0]
    #reseu = json.loads(tres)
    print res["trans_result"]
    print tres['dst']
    #print response.read()
    print stre
except Exception, e:
    print e
finally:
    if httpClient:
        httpClient.close()