WO2005071560A1 - Method and system for application performance management - Google Patents

Method and system for application performance management Download PDF

Info

Publication number
WO2005071560A1
WO2005071560A1 PCT/KR2005/000151 KR2005000151W WO2005071560A1 WO 2005071560 A1 WO2005071560 A1 WO 2005071560A1 KR 2005000151 W KR2005000151 W KR 2005000151W WO 2005071560 A1 WO2005071560 A1 WO 2005071560A1
Authority
WO
WIPO (PCT)
Prior art keywords
requests
server
tier
servers
repository
Prior art date
Application number
PCT/KR2005/000151
Other languages
French (fr)
Inventor
Keebok Park
Original Assignee
Keebok Park
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Keebok Park filed Critical Keebok Park
Publication of WO2005071560A1 publication Critical patent/WO2005071560A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/535Tracking the activity of the user
    • AHUMAN NECESSITIES
    • A44HABERDASHERY; JEWELLERY
    • A44BBUTTONS, PINS, BUCKLES, SLIDE FASTENERS, OR THE LIKE
    • A44B15/00Key-rings
    • A44B15/005Fobs
    • AHUMAN NECESSITIES
    • A45HAND OR TRAVELLING ARTICLES
    • A45DHAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
    • A45D29/00Manicuring or pedicuring implements
    • A45D29/04Nail files, e.g. manually operated
    • FMECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F21LIGHTING
    • F21LLIGHTING DEVICES OR SYSTEMS THEREOF, BEING PORTABLE OR SPECIALLY ADAPTED FOR TRANSPORTATION
    • F21L4/00Electric lighting devices with self-contained electric batteries or cells
    • F21L4/005Electric lighting devices with self-contained electric batteries or cells the device being a pocket lamp
    • FMECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F21LIGHTING
    • F21VFUNCTIONAL FEATURES OR DETAILS OF LIGHTING DEVICES OR SYSTEMS THEREOF; STRUCTURAL COMBINATIONS OF LIGHTING DEVICES WITH OTHER ARTICLES, NOT OTHERWISE PROVIDED FOR
    • F21V33/00Structural combinations of lighting devices with other articles, not otherwise provided for
    • F21V33/0004Personal or domestic articles
    • F21V33/004Sanitary equipment, e.g. mirrors, showers, toilet seats or paper dispensers

Definitions

  • the present invention relates to the application performance management and more specifically to the automated tests, analyses and monitoring the performances of the applications in the multi-tiered, distributed computing environment to optimize or to predict the performances of the applications or the individual server or the entire system.
  • a method and system are needed to make tests, analyses and monitoring the performance of the each server and the entire system that is in the multi-tiered, distributed computing environment.
  • a method and system for recording every request that is requested by each client and processed by each server in the multi-tiered, distributed computing environment is needed.
  • the web client make a request (URL) to the web server and the web server relays the requested URL to the web application server and the web application server processes the application identified by the requested URL and the application makes several database requests (SQLs) to the database server.
  • SQLs database requests
  • the database server processes the requested SQLs and the responses will go to the application in the web application server and the application in the web application server will send its response to the web server and the web server finally sends the final-response to the web client.
  • a method is needed to record and save the web client's request (URL) and at the same time, to record and save web application's requests (SQLs). And more, a method is needed to record and save the sequences, the response times, the response codes of every URL and SQL that made by every client and every server in each tier.
  • the present invention is presented to solve the above-mentioned problem and an object of the present invention is to provide a method and system for recording requests of the every client and every server in each tier in the multi-tiered, distributed computing environment to make tests the performance of the servers in each tier.
  • An object of the present invention is to provide a method and system for aggregating interrelated requests into a set and saving it into the repository so that users can make performance tests with the every set of requests and make a new set of requests by combining the saved sets of requests.
  • An object of the present invention is to provide a method and system for analyzing the performances of the requests generated by an arbitrary number of virtual clients, the performances of the servers in each tier, the performances of the requests created by every server in each tier to accomplish the requests of the virtual client in the first tier.
  • virtual clients make requests with the saved sets of requests in the repository.
  • An object of the present invention is to provide a method and system for saving performance data of the tests into the repository and analyzing the data in the repository, subsequently ebables users to optimize the performances of the servers in each tier.
  • An object of the present invention is to provide a method and system for monitoring the performances of the servers in real-time, having several virtual clients replay some requests eternally.
  • the requests are selected and queried from the repository on the basis of performance effect on the monitored system.
  • the present invention to solve the above-mentioned problem comprises a controller, agents, a repository, a performance monitor, and a user interface.
  • the controller provides a job-scheduler, which manages the load tests which performed with the recorded requests of the clients, a workbench, which records the clients' requests and makes performance tests of the each request in real time, an analyzer, which analyzes the performance of the requests and servers with the performance data in the repository, a vuser, which creates a number of virtual clients and has them replay the requests and sends the result data to controller to save the data into the repository, a performance monitor, which monitors the status of the tests in real time.
  • An agent can run on any machine independent of the location of ther servers run on.
  • an agent provides a bridge, which connects to its server with the communication protocol that the server uses when communicating with its clients, a log-manager, which gathers the performance data of the clients' requests and sends the data into the repository.
  • the repository is a database management system that manages tasks, (a task is the smallest set of requests), jobs (a job is a combination of tasks), every individual client's request, the performance data of the servers.
  • the key point of the present invention is the roles of the agents of the servers in each tier in the multi-tiered, distributed computing environment.
  • Each agent accepts the client's requests in place of the servers in each tier and sends the received requests to the server that the agent is connected to with communication protocol and receives the responses from the server and then sends the responses to the client. Meanwhile, each agent saves the details of the each client's requests and the server's responses into the repository so as to use them later for analyzing the performances of the servers in each tier and for replaying the recorded requests of the clients in each tier to make performance tests.
  • an agent gathers the details of the clients' requests and the server's responses residing between the clients and the servers by supporting the protocol that the clients and the servers use when communicating with each other.
  • Each client/server in each tier must be configured to send its requests to the agent instead of its server.
  • Each agent saves the details of the clients' requests and the server's responses into the repository.
  • the tests, analyses and monitoring the performance of the clients' requests and the servers' responses in the multi-tiered, distributed computing environment make it possible to optimize and predict the performance of the individual request as well as the performance of each server in that environment.
  • the tests, analyses and monitoring the performances of the clients' requests and the servers' responses in the multi-tiered, distributed computing environment are possible independent of the application languages, the operating systems and the underlying hardware, because the agents gathers the details of the clients' requests and the servers' responses using the protocols only that clients and servers use when communicating with each other.
  • Figure 1 is a block diagram of a preferred embodiment of the present invention which comprising the controller, the agents, the repository, the performance monitor and the user interface.
  • Figure 2 is a block diagram illustration of a configuration of an embodiment of the present invention in which a database server and its clients are in the computing environment.
  • FIG. 3 is a block diagram illustration of a configuration of an embodiment of the present invention in which a database server, a middleware server, a web application server and their clients are in the computing environment (3-tierd).
  • the figure 1 is a block diagram describing a preferred embodiment of the present invention which comprises a controller(l ⁇ ), a repository(20), a performance monitor(30), agents(41,42,43), a user interface(50), which makes possible to record and manage clients' requests in every tier and to make tests, analyze, monitor the performances of the severs in every tier with the recorded clients' requests.
  • the system to make tests, analyze, monitor the performance of the servers in the multi-tiered, distributed computing environment comprises a controller(l ⁇ ), a repository(20), a performance monitor(30), agents(40) and a user interface(50).
  • the controller (10) comprises vuser(l 1), a workbench(12), a job-scheduler(13) and an analyzer(14).
  • the vuser(l 1) has means to communicate with each server in each tier.
  • the repository(20) is a database management system which supports standard SQLs.
  • the performance monitor(30) is a window application.
  • the user interface(50) is a standard web browser.
  • the agents(41,42,43) connect the servers(61,62,63) with the supported protocols respectively and may not run on the same machine that the connected servers run on and gathers the clients' requests and the servers' responses and saves them into the repository(20).
  • users can make tests, analyze and monitor the performance of the servers in the multi-tiered, distributed computing environment only if the users have a user interface(50) and optionally, the client applications(70) of the servers(60).
  • the recording clients' requests are necessary to test, analyze the performances of the servers in the multi-tiered, distributed computing environment. It is divided into two cases whether the web clients can access the servers or not. [57] In case it is possible to access the server with the web browser, it comes under the figure 1. [58] In this case, users navigate to accomplish a certain tasks with the user interface (50, web browser), the workbench (12), the web agent (41) and the web application server (61). [59] The web agent (41) gathers the performance data and saves them into the repository (20), here the performance data are requested URLs, the time intervals between the requests (stay times), the response times, response codes, response sizes, http request headers, http response headers and cookies for example.
  • the web application server(61) which connected to the middleware server(62) by the middleware agent(42) makes RPC(remote procedure call) requests to middleware agent(42) to accomplish the URLs requested by the web clients.
  • the middleware agent(42) gathers the performance data of the PRC requests and saves them into the repository (20).
  • users create tasks by querying the recorded RPC requests from the repository(20) with the user interface(50) through the workbench(12).
  • the middleware server(62) which connected to the database server(63) by the database agent(43) makes SQL requests to the database agent(43) to accomplish the RPC requests requested by the web application server (61).
  • the database agent(43) gathers the performance data of the SQL requests and saves them into the repository (20).
  • users create tasks by querying the recorded SQL requests from the repository(20) with the user interface(50) through the workbench(12).
  • users accomplish a certain tasks with the client applications of the server.
  • the database agent(43) gathers the performance data and saves them into the repository (20), here the performance data are requested SQLs, the time intervals between the requests (stay times), the response times, response codes and response sizes.
  • http request headers such as query string, cookie, referer, user-agent, authentication and content-type as well as the requested URL itself, iteration options and stay times are saved in the repository (20).
  • the file('data.txt') function replaces its value with a line data from the data.txt file while loops.
  • the job-scheduler(13) make performance tests according to a job which is combined with the tasks. Setting iteration option to each tasks when combining tasks makes it possible to create a variety of performance test scenarios. All the tasks and jobs can be reused and commonly used by all users because they are all saved and managed in the repository(20).
  • the web agent(41) saves the performance data of the requested URL into the repository(20), which processed in the web application server(61).
  • the middleware agent(42) saves the performance data of the requested RPC requests into the repository (20), which are requested by the web application server(61) to the middleware server(62) to accomplish its(61) requested URL.
  • the database agent(43) saves the performance data of the requested SQLs into the repository(20), which are requested by the middleware server(62) to the database server(63) to accomplish its(62) requested RPC requests.
  • the workbench(12) analyze the sequence, the response times, the number of times of each request and the contents of the URL, SQLs, RPC requests that processed in the servers in each tier with the help of analyzer(14) (which can access the repository(20) ) and sends the data to the user interface(50).
  • the web-tier is not the first-tier, it is possible to analyze the sequence, the response times, the response sizes, etc of the requests which are induced and processed in all servers.
  • each agent of each server will gather the performance data of the requests and saves the data into the repository. So, users can query the performance data of the every server from the repository with the user interface. With this method, users can test, analyze and monitor the performance of other servers just with a testing to a server.
  • web application server(61) will make RPC requests to the middleware agent(42) to accomplish the requested URLs.
  • the middleware agents(42) will sends the RPC requests to the middleware server(62), and receives the responses from the middleware server(62) and saves the performance data into the repository (20).
  • middleware server(62) will make SQL requests to the database agent(43) to accomplish the requested RPC requests.
  • the database agents(43) will sends the SQL requests to the database server(63), and receives the responses from the database server(63) and saves the performance data into the repository (20).
  • the vuser(l 1) will save the performance data of the requested URLs into the repository (20) after receiving the responses.
  • the performance monitor(30) running inside the user interface(50) makes it possible to monitor the performances of the servers in real time through receiving performance data from the controller(30).
  • the controller(30) receives the performance data from the agents(40), the vuser(l 1) and the servers(60) (including system resource usage information).
  • FIG 1 there are web application server(61), middleware server(62) and database server(63).
  • the vuser(l 1) changes its status to be exclusively locked and creates a virtual client and has the virtual client test with each URLs to analyze the sequences, the response times, the response sizes, etc of each URL and induced RPC calls and SQLs and then saves the data into the repository (20). And then, the vuser(l 1) changes its status to be normal mode and make performance test with a number of virtual clients. In the meanwhile, each agent(40) in each tier saves the performance data of the requested URLs, RPC calls and SQLs into the repository (20).
  • users can query the sequences, the response times, the response sizes, etc of each requested URL, RPC call, SQL and can query the performance data of all the servers as well.
  • the performance data can be queried by time, by day, by month and by year.
  • the top/bottom-N ranked queries on the performance data such as response times, response sizes and bytes per second can be performed from the repository(20).
  • the present invention can be used in the multi-tiered, distributed computing environment in which following servers are running.

Abstract

The present invention relates to the methods and system for recording clients' requests and for making tests, analyses and monitoring the performances of the requests and the performances of the servers in the multi-tiered, distributed computing environment. More specifically, running each agent for each server, each agent relaying its client's requests and its server's responses between the client and server to record clients' requests and for making automated tests, analyses and monitoring, with the recorded requests, the performances of the requests and the induced requests by the servers and the performances of the servers in the multi-tiered, distributed computing environment.

Description

Description METHOD AND SYSTEM FOR APPLICATION PERFORMANCE MANAGEMENT Technical Field
[I] The present invention relates to the application performance management and more specifically to the automated tests, analyses and monitoring the performances of the applications in the multi-tiered, distributed computing environment to optimize or to predict the performances of the applications or the individual server or the entire system. Background Art
[2] Today's computing environment has grown from client-server computing environment to the multi-tiered, distributed computing one in which web servers, database servers, middleware servers and server applications play their roles to accomplish the client's requests. The servers in this environment will cooperate with one another to accomplish the client's requests.
[3] Generally, tests, analyses and monitoring the performance of the web application systems are performed according to the following process.
[4] Recording and saving a web client's requests (URL, Uniform Resource Locator) in order in which they were requested.
[5] And creating virtual web clients and have them replay the recorded requests.
[6] And analyzing the virtual web clients' responses and the performances of the web application systems.
[7] Here, in order to record and save the web client's requests, a client application is needed which can communicate with web application servers and can record and save the requests in order in which they were requested.
[8] Generally, tests, analyses and monitoring the performance of the database systems are performed according to the following process.
[9] Creating a database client's requests (SQLs, Structured Query Language) manually in order in which the real database client requested them.
[10] And creating virtual database clients and have them replay the created requests. And Analyzing the virtual database clients' responses and the performances of the database systems.
[II] Here, a database client's requests are created manually, not automatically, in most cases.
[12] These methods to make tests, analyses and monitoring the performance of the servers are not easily applicable to the systems in the multi-tiered, distributed computing environment. Methods and system are needed to make tests, analyses and monitoring the performance of the each server in the multi-tiered, distributed computing environment and to optimize and predict the performance of the entire system. Disclosure of Invention Technical Problem
[13] A method and system are needed to make tests, analyses and monitoring the performance of the each server and the entire system that is in the multi-tiered, distributed computing environment. A method and system for recording every request that is requested by each client and processed by each server in the multi-tiered, distributed computing environment is needed. For example, in a system which is comprised of a web client, a web server, a web application server and a database server, the web client make a request (URL) to the web server and the web server relays the requested URL to the web application server and the web application server processes the application identified by the requested URL and the application makes several database requests (SQLs) to the database server. The database server processes the requested SQLs and the responses will go to the application in the web application server and the application in the web application server will send its response to the web server and the web server finally sends the final-response to the web client. Here, a method is needed to record and save the web client's request (URL) and at the same time, to record and save web application's requests (SQLs). And more, a method is needed to record and save the sequences, the response times, the response codes of every URL and SQL that made by every client and every server in each tier.
[14] The information of the recorded URLs and SQLs that requested by each client and processed by each server makes it possible to analyze the performance of the servers in each tier. And the information also makes it possible to make tests and monitoring to analyze and optimize each server in each tier.
[15] The present invention is presented to solve the above-mentioned problem and an object of the present invention is to provide a method and system for recording requests of the every client and every server in each tier in the multi-tiered, distributed computing environment to make tests the performance of the servers in each tier.
[16] An object of the present invention is to provide a method and system for aggregating interrelated requests into a set and saving it into the repository so that users can make performance tests with the every set of requests and make a new set of requests by combining the saved sets of requests.
[17] An object of the present invention is to provide a method and system for analyzing the performances of the requests generated by an arbitrary number of virtual clients, the performances of the servers in each tier, the performances of the requests created by every server in each tier to accomplish the requests of the virtual client in the first tier. Here, virtual clients make requests with the saved sets of requests in the repository.
[18] An object of the present invention is to provide a method and system for saving performance data of the tests into the repository and analyzing the data in the repository, subsequently ebables users to optimize the performances of the servers in each tier.
[19] An object of the present invention is to provide a method and system for monitoring the performances of the servers in real-time, having several virtual clients replay some requests eternally. Here, the requests are selected and queried from the repository on the basis of performance effect on the monitored system. Technical Solution
[20] The present invention to solve the above-mentioned problem comprises a controller, agents, a repository, a performance monitor, and a user interface. Specifically, the controller provides a job-scheduler, which manages the load tests which performed with the recorded requests of the clients, a workbench, which records the clients' requests and makes performance tests of the each request in real time, an analyzer, which analyzes the performance of the requests and servers with the performance data in the repository, a vuser, which creates a number of virtual clients and has them replay the requests and sends the result data to controller to save the data into the repository, a performance monitor, which monitors the status of the tests in real time. An agent can run on any machine independent of the location of ther servers run on. And an agent provides a bridge, which connects to its server with the communication protocol that the server uses when communicating with its clients, a log-manager, which gathers the performance data of the clients' requests and sends the data into the repository. The repository is a database management system that manages tasks, (a task is the smallest set of requests), jobs (a job is a combination of tasks), every individual client's request, the performance data of the servers.
[21] The key point of the present invention is the roles of the agents of the servers in each tier in the multi-tiered, distributed computing environment. Each agent accepts the client's requests in place of the servers in each tier and sends the received requests to the server that the agent is connected to with communication protocol and receives the responses from the server and then sends the responses to the client. Meanwhile, each agent saves the details of the each client's requests and the server's responses into the repository so as to use them later for analyzing the performances of the servers in each tier and for replaying the recorded requests of the clients in each tier to make performance tests. [22] In this way, an agent gathers the details of the clients' requests and the server's responses residing between the clients and the servers by supporting the protocol that the clients and the servers use when communicating with each other. [23] Each client/server in each tier must be configured to send its requests to the agent instead of its server. Each agent saves the details of the clients' requests and the server's responses into the repository. [24] Advantageous Effects
[25] According to the present invention, the tests, analyses and monitoring the performance of the clients' requests and the servers' responses in the multi-tiered, distributed computing environment make it possible to optimize and predict the performance of the individual request as well as the performance of each server in that environment.
[26] And according to the present invention, the tests, analyses and monitoring the performances of the clients' requests and the servers' responses in the multi-tiered, distributed computing environment are possible independent of the application languages, the operating systems and the underlying hardware, because the agents gathers the details of the clients' requests and the servers' responses using the protocols only that clients and servers use when communicating with each other.
[27] And according to the present invention, it is possible to reuse the test conditions and scenarios and analyze, predict the performances of the requests and the performances of the servers in detail, because the tasks, jobs (which are the test conditions and scenarios) and performance data are all saved and managed in the repository database.
[28] And according to the present invention, it is possible to monitor constantly the performance of the servers under operation in the multi-tiered, distributed computing environment without much load on the servers by making tests constantly on the servers with a job in which tasks are comprised of the selected requests that having critical effects on the performance of the servers. Brief Description of the Drawings
[29] Figure 1 is a block diagram of a preferred embodiment of the present invention which comprising the controller, the agents, the repository, the performance monitor and the user interface.
[30] Figure 2 is a block diagram illustration of a configuration of an embodiment of the present invention in which a database server and its clients are in the computing environment.
[31] Figure 3 is a block diagram illustration of a configuration of an embodiment of the present invention in which a database server, a middleware server, a web application server and their clients are in the computing environment (3-tierd).
[32] <Names of the blocks in the figures>
[33] 10. Controller
[34] 11. VUser 12. Workbench 13. Job-Scheduler 14. Analyzer
[35] 20. Repository
[36] 30. Performance Monitor
[37] 40. Agents
[38] 41. Web Agent 42. Middleware Agent 43. Database Agent
[39] 50. User Interface
[40] 60. Servers
[41] 61. Web Application Server 62. Middleware Server 63. Database Server
[42] 70. Client Applications
[43] 71. Web Client 72. Middleware Client 73. Database Client
[44] Best Mode for Carrying Out the Invention
[45] A preferred embodiment of the present invention will now be described, by way of example, with reference to the accompanying drawings.
[46] The figure 1 is a block diagram describing a preferred embodiment of the present invention which comprises a controller(lθ), a repository(20), a performance monitor(30), agents(41,42,43), a user interface(50), which makes possible to record and manage clients' requests in every tier and to make tests, analyze, monitor the performances of the severs in every tier with the recorded clients' requests.
[47] As shown in figure 1, the system to make tests, analyze, monitor the performance of the servers in the multi-tiered, distributed computing environment according to the present invention comprises a controller(lθ), a repository(20), a performance monitor(30), agents(40) and a user interface(50).
[48] The controller (10) comprises vuser(l 1), a workbench(12), a job-scheduler(13) and an analyzer(14).
[49] The vuser(l 1) has means to communicate with each server in each tier.
[50] The repository(20) is a database management system which supports standard SQLs.
[51] The performance monitor(30) is a window application.
[52] The user interface(50) is a standard web browser.
[53] The agents(41,42,43) connect the servers(61,62,63) with the supported protocols respectively and may not run on the same machine that the connected servers run on and gathers the clients' requests and the servers' responses and saves them into the repository(20). [54] With the system that created according to the present invention, users can make tests, analyze and monitor the performance of the servers in the multi-tiered, distributed computing environment only if the users have a user interface(50) and optionally, the client applications(70) of the servers(60). [55] <Recording and managing clients's requests>
[56] Firstly, the recording clients' requests are necessary to test, analyze the performances of the servers in the multi-tiered, distributed computing environment. It is divided into two cases whether the web clients can access the servers or not. [57] In case it is possible to access the server with the web browser, it comes under the figure 1. [58] In this case, users navigate to accomplish a certain tasks with the user interface (50, web browser), the workbench (12), the web agent (41) and the web application server (61). [59] The web agent (41) gathers the performance data and saves them into the repository (20), here the performance data are requested URLs, the time intervals between the requests (stay times), the response times, response codes, response sizes, http request headers, http response headers and cookies for example. [60] And users create tasks by querying the recorded URLs from the repository(20) with the user interface(50) through the workbench(12). Here, the tasks are the sets of requests and the basic unit of test scenarios. [61] In the mean time, for the middleware tier, the web application server(61) which connected to the middleware server(62) by the middleware agent(42) makes RPC(remote procedure call) requests to middleware agent(42) to accomplish the URLs requested by the web clients. [62] The middleware agent(42) gathers the performance data of the PRC requests and saves them into the repository (20). [63] And users create tasks by querying the recorded RPC requests from the repository(20) with the user interface(50) through the workbench(12). [64] And for the database tier, the middleware server(62) which connected to the database server(63) by the database agent(43) makes SQL requests to the database agent(43) to accomplish the RPC requests requested by the web application server (61). [65] The database agent(43) gathers the performance data of the SQL requests and saves them into the repository (20). [66] And users create tasks by querying the recorded SQL requests from the repository(20) with the user interface(50) through the workbench(12). [67] In case it is impossible to access the server with the web browser, users accomplish a certain tasks with the client applications of the server.
[68] In a configuration in which a database server and its clients are in the computing environment, as shown in figure 2, for example, the database agent(43) gathers the performance data and saves them into the repository (20), here the performance data are requested SQLs, the time intervals between the requests (stay times), the response times, response codes and response sizes.
[69] And users create tasks by querying the recorded SQL requests from the repository(20) with the user interface(50) through the workbench(12).
[70] The sets of the client's requests to the servers in each tier are saved in the repository(20) as tasks. The IP addresses of the servers, iteration options, time intervals between the two requests as well as the requests themselves are saved in the repository (20).
[71] For http, http request headers such as query string, cookie, referer, user-agent, authentication and content-type as well as the requested URL itself, iteration options and stay times are saved in the repository (20).
[72] Applying some functions on the properties of the clients' requests makes it possible to make a variety of tests with varying conditions, the functions are sequence(F,T,I), file('data.txt') and random(F,T).
[73] The sequence(F,T,I) function replaces its value with a number from F to T incremented or decremented by I while loops.
[74] The file('data.txt') function replaces its value with a line data from the data.txt file while loops.
[75] The random(F,T) function replaces its value with a random number between F and T while loops.
[76] The job-scheduler(13) make performance tests according to a job which is combined with the tasks. Setting iteration option to each tasks when combining tasks makes it possible to create a variety of performance test scenarios. All the tasks and jobs can be reused and commonly used by all users because they are all saved and managed in the repository(20).
[77] <Analyzing the sequences, the response times, the response sizes, etc. of all the requests processed in each server, which are induced by the first-tier client's requests to the first-tier server>
[78] With the help of the performance data of the requests and responses gathered by each agent(40) in repository(20), users can analyze the sequences, the response times, the response sizes, etc. of all the requests processed in each server, which are induced by the first-tier client's requests to the first-tier server.
[79] In figure 1, let the web-tier be the first-tier, so that web clients can communicate with the entire system by accessing the web-tier server. [80] Users make performance test with a URL that selected from the recorded URLs from the repository(20) with the user interface(50) and the workbench(12).
[81] The web agent(41) saves the performance data of the requested URL into the repository(20), which processed in the web application server(61).
[82] The middleware agent(42) saves the performance data of the requested RPC requests into the repository (20), which are requested by the web application server(61) to the middleware server(62) to accomplish its(61) requested URL.
[83] The database agent(43) saves the performance data of the requested SQLs into the repository(20), which are requested by the middleware server(62) to the database server(63) to accomplish its(62) requested RPC requests.
[84] And finally, the workbench(12) analyze the sequence, the response times, the number of times of each request and the contents of the URL, SQLs, RPC requests that processed in the servers in each tier with the help of analyzer(14) ( which can access the repository(20) ) and sends the data to the user interface(50).
[85] In the meanwhile, the workbench(12) and all the agents(41,42,43) are exclusively locked, so that it is guaranteed that only one user could make performance test without interruptions of the other concurrent users.
[86] And more, using the same method described above, although the web-tier is not the first-tier, it is possible to analyze the sequence, the response times, the response sizes, etc of the requests which are induced and processed in all servers.
[87] <Analyzing the performances of other servers with a performance test to a server>
[88] It is possible to make tests the performance of a server with the user interface(50), a set of recorded requests, having created virtual clients load on the server and analyze, monitor the performance of the server and the performances of the requests.
[89] Here, if the tested server makes additional requests to other servers to accomplish its requests, each agent of each server will gather the performance data of the requests and saves the data into the repository. So, users can query the performance data of the every server from the repository with the user interface. With this method, users can test, analyze and monitor the performance of other servers just with a testing to a server.
[90] The above described can be completely understood through the detailed description of an embodiment below and the accompanying drawings;
[91] In figure 1, lets assume the above tested server be web application server(61), and the URLs are already recorded and saved in the repository(20).
[92] Scheduling a performance test to the web application server(61) with the user interface(50) and the job-scheduler(13), the vuser(ll) will create a number of virtual clients and have them make URL requests to the web application server(61).
[93] Then, web application server(61) will make RPC requests to the middleware agent(42) to accomplish the requested URLs.
[94] And then, the middleware agents(42) will sends the RPC requests to the middleware server(62), and receives the responses from the middleware server(62) and saves the performance data into the repository (20).
[95] Meanwhile, middleware server(62) will make SQL requests to the database agent(43) to accomplish the requested RPC requests.
[96] And then, the database agents(43) will sends the SQL requests to the database server(63), and receives the responses from the database server(63) and saves the performance data into the repository (20).
[97] And then, the vuser(l 1) will save the performance data of the requested URLs into the repository (20) after receiving the responses.
[98] Subsequently, in the repository(20), there saved the performance data of all the requests (URLs, RPC calls, SQLs) that processed in the servers. So, users can analyze the performances of the URLs, RPC calls, SQLs by querying the performance data from the repository (20).
[99] And the performance monitor(30) running inside the user interface(50) makes it possible to monitor the performances of the servers in real time through receiving performance data from the controller(30). Here, the controller(30) receives the performance data from the agents(40), the vuser(l 1) and the servers(60) (including system resource usage information).
[100] < Analyzing the performance of every request that induced and processed in each server with a performance test to a server>
[101] As described above, it is possible to analyze the sequences, the response times, the response sizes, the number of times, etc of the each request that induced and processed in each server using single virtual client and it is also possible to analyze the performances of other servers with a performance test to a server using a number of virtual clients.
[102] With the above two analyses , it is possible to analyze the sequences, the response times, the response sizes, the number of times, etc of the each request that induced and processed in each server as well as the performances of other servers with a performance test to a server using a number of virtual clients.
[103] The above described can be completely understood through the detailed description of an embodiment below and the accompanying drawings;
[104] In figure 1, there are web application server(61), middleware server(62) and database server(63).
[105] If users make performance tests to the web application server(61) with a number of virtual clients, the vuser(l 1) changes its status to be exclusively locked and creates a virtual client and has the virtual client test with each URLs to analyze the sequences, the response times, the response sizes, etc of each URL and induced RPC calls and SQLs and then saves the data into the repository (20). And then, the vuser(l 1) changes its status to be normal mode and make performance test with a number of virtual clients. In the meanwhile, each agent(40) in each tier saves the performance data of the requested URLs, RPC calls and SQLs into the repository (20).
[106] And then, users can query the sequences, the response times, the response sizes, etc of each requested URL, RPC call, SQL and can query the performance data of all the servers as well.
[107] <Managing, analyzing, and querying the performance data>
[108] Users can query a variety of information from the repository (20) with the user interface(50) through the analyzer(14), because all the performance data that tested and gathered are saved in the repository (20).
[109] Explaining this in more detail, the performance data can be queried by time, by day, by month and by year. The top/bottom-N ranked queries on the performance data such as response times, response sizes and bytes per second can be performed from the repository(20).
[110] It is possible to query performance data based on varying test conditions, the number of virtual clients, the staying times and the persistence options. And it is also possible to merge queried results into a new one so that users can compare and analyze the performances of the servers.
[Ill] <Monitoring and analyzing the systems under operation>
[112] Explaining this in detail along with the figure 3, having job-scheduler(13) and vuser(l 1) make performance tests constantly with the requests queried from the repository(20) based on the performance effect on the servers(60). Then, it is possible to monitor the performances of the servers(60) by the performances of the selected requests in real time. And because all the performance data are saved in the repository, it is possible to analyze and monitor in detail the performance of the systems(60) under operation with the user interface(50) and the analyzer(14).
[113] Mode for the Invention
[114] Although above-described embodiments are particular examples of the methods and system for application performance management, it will be understood that the present invention is not limited thereto and many modifications and additions are possible within the scope of the present invention's technology.
[115] For example, the present invention can be used in the multi-tiered, distributed computing environment in which following servers are running. Xml database servers, web servers, ERP solutions, CRM(Customer Relationship Management) solutions that support HTTP(HTTPS) protocol and CORBA(Common Object Request Broker Architecture) middleware that supports IIOP(Internet Inter-Object Protocol) and EJB (Enterprise Java Beans) servers that supports RMI(Remote Method Invocation) protocol and any servers that support FTP(File Transfer Protocol), TELNET, SMTP(Simple Mail Transfer Protocol), POP3(Post Office Protocol 3), IMAP(Internet Message Access Protocol), NNTP(Network News Transfer Protocol), MMS(Microsoft Media Service), RTSP(Real Time Streaming Protocol), SOCKS(Firewall Protocol) protocols.
[116] Industrial Applicability
[117] According to the methods and system of the present invention, it is possible to optimize the performance of the developing system through the tests, analyses the performances of the systems when companies are developing their information systems, internet shopping malls or web sites. And it is possible to predict the performances of the systems before it starts to service officially through a variety of tests and analyses. Making a company's IT systems have better performance, lowers costs of management of the systems, of course it is beneficial to the company. The monitoring the performances of the servers constantly in the multi-tiered, distributed computing environment makes the entire system have high performance and high availability.

Claims

Claims
[1] A method for recording the URLs, RPC calls, SQLs that requested and processed by the servers in the multi-tiered, distributed computing environment to use later for performance tests or for replaying the communications, said method comprising the steps of; Running agents for each server to relay the requests and the responses between the clients and the server; Setting the the configurations of the applications with the IP addresses and port numbers of the agents instead of the those of the servers, the applications are running on the servers, so that the applications' requests go to the agents, not to the servers; Using the user interface or the client's applications, connecting to the first-tier agent and making requests to the agent that connects the first-tier server that will service to the client's request; The first-tier agent receiving the client's request, sending the request to the connected, the first-tier server, receiving the response from the first-tier server and saving the performance data of the request and the response into the repository; The first-tier server making additional requests to the second-tier agent to accomplish the first-tier client's request, the second-tier agent receiving the requests from the first-tier server and sending the requests to the second-tier server that is connected by the second-tier agent; The second-tier agent receiving the first-tier server's requests, sending the requests to the connected the second-tier server, receiving the response from the second-tier server and saving the performance data of the requests and the responses to the repository; In this way, having each agent for each server save the performance data of the requests that made by the clients and each server and the responses that processed by each server into the repository;
[2] A method for analyzing the performance data such as the sequence, the response times and the response sizes of the requests, where the requests are typically URLs, RPC calls and SQLs that requested by the clients and the servers in the multi-tiered, distributed computing environment, said method comprising the steps of; Running agents for each server to relay the requests and the responses between the clients and the server; Setting the the configurations of the applications with the IP addresses and port numbers of the agents instead of the those of the servers, the applications are running on the servers, so that the applications' requests go to the agents, not to the servers; Using the user interface or the client's applications, connecting to the first-tier agent and making requests to the agent that connects the first-tier server that will service to the client's request; The first-tier agent receiving the client's request, sending the request to the connected, the first-tier server, receiving the response from the first-tier server and saving the performance data of the request and the response into the repository; The first-tier server making additional requests to the second-tier agent to accomplish the first-tier client's request, the second-tier agent receiving the requests from the first-tier server and sending the requests to the second-tier server that is connected by the second-tier agent; The second-tier agent receiving the first-tier server's requests, sending the requests to the connected the second-tier server, receiving the response from the second-tier server and saving the performance data of the requests and the responses to the repository; In this way, having each agent for each server save the performance data of the requests that made by the clients and each server and the responses that processed by each server into the repository, where the requests are typically URLs, RPC calls and SQLs; By querying the performance data from the repository, analyzing the sequences, the response times, the response sizes of the requests that processed by each server. [3] A method for analyzing the performance metrics of the servers such as mean response times, transactions per second, bytes per second and the performances of the requests such response times, response sizes using virtual clients, user interface and the sets of clients' requests (typically sets of URLs, RPC calls, SQLs) in the multi-tiered, distributed computing environment, said method comprising the steps of; Running agents for each server to relay the requests and the responses between the clients and the server; Setting the the configurations of the applications with the IP addresses and port numbers of the agents instead of the those of the servers, the applications are running on the servers, so that the applications' requests go to the agents, not to the servers; Preparing for the load test by creating tasks and jobs that are the test scenarios, using the user interface and job-scheduler, Using the ability of the virtual clients to make requests(URLs, RPC calls, SQLs) to the servers, having the virtual clients perform the load test according to the test scenarios described in the created job. The tested server making requests to an agent to accomplish the virtual clients' requests, the agent sending the requests to a server that is connected by the agent; The agent receiving the virtual clients' requests, sending the requests to the connected the server, receiving the responses from the server and saving the performance data of the requests and the responses into the repository; In this way, having each agent for each server save the performance data of the requests and the responses that requested and processed by each server into the repository, where the requests are typically URLs, RPC calls and SQLs; The virtual clients receiving the responses from the tested server and saving the performance data of the requests and the responses to the repository; By querying the performance data from the repository, users can analyze the performances of the servers such as mean response times, transactions per second, bytes per second and the performance of the requests such as the sequences, the response times, the response sizes of the requests that processed by each server. [4] A system for making tests, analyses and monitoring the performances of the servers in the multi-tired, distributed computing environment, the system comprising; A controlled 10), agents(40), a repository(20), a performance monitor(30) and a user interface(50); The controller (10) comprising; Means(l 1) for creating a number of virtual clients and communicating with the servers in each tier; means(12) for making performance tests under exclusive lock and communicating with the agents(40) to analyze the performances of the requests and servers in each tier, means(13) for scheduling the test scenarios, the jobs with various test conditions; means for communicating the performance data with the performance monitor(30); means for querying the recorded requests from the repository (20); means(13) for creating a test scenario (a job) combining the recorded set of requests (tasks) from the repository (20); means(14) for communicating with the repository (20) and querying the performance data from the repository (20); The agents(40) comprising; means for communicating with the connected server with the supported communication protocol; means for gathering the performance data of the client's requests that are requested to the connected server and saving the data into the repository (20); means for making exclusive lock status for only a client communicating with the agent;
The repository (20) comprising; means for communicating with the controlled 10) and saving the performance data; means for communicating with the agents(40) and saving the performance data; The performance monitor(30); means for communicating with the controlled 10) and receiving the performance data of the servers in each tier; means for displaying the performance data of the servers which received from the controlled 10); The user interface(50) comprising; means for users communicating with the controlled 10), the agents(40) and the performance monitor(30);
PCT/KR2005/000151 2004-01-17 2005-01-15 Method and system for application performance management WO2005071560A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2004-0003589 2004-01-17
KR10-2004-0003589A KR100447041B1 (en) 2004-01-17 2004-01-17 Method and system for making performance test scenarios and for testing, analyzing and motoring the performance of servers in multi-tier distributed computing environment.

Publications (1)

Publication Number Publication Date
WO2005071560A1 true WO2005071560A1 (en) 2005-08-04

Family

ID=34805994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2005/000151 WO2005071560A1 (en) 2004-01-17 2005-01-15 Method and system for application performance management

Country Status (2)

Country Link
KR (1) KR100447041B1 (en)
WO (1) WO2005071560A1 (en)

Cited By (85)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7685272B2 (en) 2006-01-13 2010-03-23 Microsoft Corporation Application server external resource monitor
US8433771B1 (en) * 2009-10-02 2013-04-30 Amazon Technologies, Inc. Distribution network with forward resource propagation
US9009676B2 (en) 2010-04-02 2015-04-14 International Business Machines Corporation Common recording and replaying performance test tool for a client-server environment
WO2015103122A3 (en) * 2013-12-31 2015-08-20 Clicktale Ltd. A method and system for tracking and gathering multivariate testing data
US9130756B2 (en) 2009-09-04 2015-09-08 Amazon Technologies, Inc. Managing secure content in a content delivery network
US9154551B1 (en) 2012-06-11 2015-10-06 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9160703B2 (en) 2010-09-28 2015-10-13 Amazon Technologies, Inc. Request routing management based on network components
US9185012B2 (en) 2010-09-28 2015-11-10 Amazon Technologies, Inc. Latency measurement in resource requests
US9191338B2 (en) 2010-09-28 2015-11-17 Amazon Technologies, Inc. Request routing in a networked environment
US9191458B2 (en) 2009-03-27 2015-11-17 Amazon Technologies, Inc. Request routing using a popularity identifier at a DNS nameserver
US9208097B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Cache optimization
US9210235B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Client side cache management
US9237114B2 (en) 2009-03-27 2016-01-12 Amazon Technologies, Inc. Managing resources in resource cache components
US9251112B2 (en) 2008-11-17 2016-02-02 Amazon Technologies, Inc. Managing content delivery network service providers
US9253065B2 (en) 2010-09-28 2016-02-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9294391B1 (en) 2013-06-04 2016-03-22 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9323577B2 (en) 2012-09-20 2016-04-26 Amazon Technologies, Inc. Automated profiling of resource usage
US9332078B2 (en) 2008-03-31 2016-05-03 Amazon Technologies, Inc. Locality based content distribution
US9391949B1 (en) 2010-12-03 2016-07-12 Amazon Technologies, Inc. Request routing processing
US9407681B1 (en) 2010-09-28 2016-08-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9407699B2 (en) 2008-03-31 2016-08-02 Amazon Technologies, Inc. Content management
US9444759B2 (en) 2008-11-17 2016-09-13 Amazon Technologies, Inc. Service provider registration by a content broker
US9451046B2 (en) 2008-11-17 2016-09-20 Amazon Technologies, Inc. Managing CDN registration by a storage provider
US9479476B2 (en) 2008-03-31 2016-10-25 Amazon Technologies, Inc. Processing of DNS queries
US9497259B1 (en) 2010-09-28 2016-11-15 Amazon Technologies, Inc. Point of presence management in request routing
US9495338B1 (en) 2010-01-28 2016-11-15 Amazon Technologies, Inc. Content distribution network
US9515949B2 (en) 2008-11-17 2016-12-06 Amazon Technologies, Inc. Managing content delivery network service providers
US9525659B1 (en) 2012-09-04 2016-12-20 Amazon Technologies, Inc. Request routing utilizing point of presence load information
US9544394B2 (en) 2008-03-31 2017-01-10 Amazon Technologies, Inc. Network resource identification
US9571389B2 (en) 2008-03-31 2017-02-14 Amazon Technologies, Inc. Request routing based on class
US9608957B2 (en) 2008-06-30 2017-03-28 Amazon Technologies, Inc. Request routing using network computing components
US9628554B2 (en) 2012-02-10 2017-04-18 Amazon Technologies, Inc. Dynamic content delivery
US9712484B1 (en) 2010-09-28 2017-07-18 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US9734472B2 (en) 2008-11-17 2017-08-15 Amazon Technologies, Inc. Request routing utilizing cost information
US9742795B1 (en) 2015-09-24 2017-08-22 Amazon Technologies, Inc. Mitigating network attacks
US9774619B1 (en) 2015-09-24 2017-09-26 Amazon Technologies, Inc. Mitigating network attacks
US9787775B1 (en) 2010-09-28 2017-10-10 Amazon Technologies, Inc. Point of presence management in request routing
US9794281B1 (en) 2015-09-24 2017-10-17 Amazon Technologies, Inc. Identifying sources of network attacks
US9800539B2 (en) 2010-09-28 2017-10-24 Amazon Technologies, Inc. Request routing management based on network components
US9819567B1 (en) 2015-03-30 2017-11-14 Amazon Technologies, Inc. Traffic surge management for points of presence
US9832141B1 (en) 2015-05-13 2017-11-28 Amazon Technologies, Inc. Routing based request correlation
US9887931B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US9887932B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US9912740B2 (en) 2008-06-30 2018-03-06 Amazon Technologies, Inc. Latency measurement in resource requests
US9930131B2 (en) 2010-11-22 2018-03-27 Amazon Technologies, Inc. Request routing processing
US9954934B2 (en) 2008-03-31 2018-04-24 Amazon Technologies, Inc. Content delivery reconciliation
US9985927B2 (en) 2008-11-17 2018-05-29 Amazon Technologies, Inc. Managing content delivery network service providers by a content broker
US9992086B1 (en) 2016-08-23 2018-06-05 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US9992303B2 (en) 2007-06-29 2018-06-05 Amazon Technologies, Inc. Request routing utilizing client location information
US10015237B2 (en) 2010-09-28 2018-07-03 Amazon Technologies, Inc. Point of presence management in request routing
US10021179B1 (en) 2012-02-21 2018-07-10 Amazon Technologies, Inc. Local resource delivery network
US10027582B2 (en) 2007-06-29 2018-07-17 Amazon Technologies, Inc. Updating routing information based on client location
US10033691B1 (en) 2016-08-24 2018-07-24 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US10033627B1 (en) 2014-12-18 2018-07-24 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10049051B1 (en) 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10075551B1 (en) 2016-06-06 2018-09-11 Amazon Technologies, Inc. Request management for hierarchical cache
US10091096B1 (en) 2014-12-18 2018-10-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10097448B1 (en) 2014-12-18 2018-10-09 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10097566B1 (en) 2015-07-31 2018-10-09 Amazon Technologies, Inc. Identifying targets of network attacks
US10110694B1 (en) 2016-06-29 2018-10-23 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US10205698B1 (en) 2012-12-19 2019-02-12 Amazon Technologies, Inc. Source-dependent address resolution
US10225326B1 (en) 2015-03-23 2019-03-05 Amazon Technologies, Inc. Point of presence based data uploading
US10230819B2 (en) 2009-03-27 2019-03-12 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10257307B1 (en) 2015-12-11 2019-04-09 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10270878B1 (en) 2015-11-10 2019-04-23 Amazon Technologies, Inc. Routing for origin-facing points of presence
US10348639B2 (en) 2015-12-18 2019-07-09 Amazon Technologies, Inc. Use of virtual endpoints to improve data transmission rates
US10372499B1 (en) 2016-12-27 2019-08-06 Amazon Technologies, Inc. Efficient region selection system for executing request-driven code
US10447648B2 (en) 2017-06-19 2019-10-15 Amazon Technologies, Inc. Assignment of a POP to a DNS resolver based on volume of communications over a link between client devices and the POP
US10469513B2 (en) 2016-10-05 2019-11-05 Amazon Technologies, Inc. Encrypted network addresses
US10503613B1 (en) 2017-04-21 2019-12-10 Amazon Technologies, Inc. Efficient serving of resources during server unavailability
US10521348B2 (en) 2009-06-16 2019-12-31 Amazon Technologies, Inc. Managing resources using resource expiration data
US10592578B1 (en) 2018-03-07 2020-03-17 Amazon Technologies, Inc. Predictive content push-enabled content delivery network
US10601767B2 (en) 2009-03-27 2020-03-24 Amazon Technologies, Inc. DNS query processing based on application information
US10616179B1 (en) 2015-06-25 2020-04-07 Amazon Technologies, Inc. Selective routing of domain name system (DNS) requests
US10623408B1 (en) 2012-04-02 2020-04-14 Amazon Technologies, Inc. Context sensitive object management
CN111324545A (en) * 2020-03-13 2020-06-23 中国工商银行股份有限公司 Application server middleware performance test method, test terminal and application server
US10831549B1 (en) 2016-12-27 2020-11-10 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10862852B1 (en) 2018-11-16 2020-12-08 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US20210056004A1 (en) * 2019-08-22 2021-02-25 Micro Focus Llc Regression testing of computer systems using recorded prior computer system communications
US10938884B1 (en) 2017-01-30 2021-03-02 Amazon Technologies, Inc. Origin server cloaking using virtual private cloud network environments
US10958501B1 (en) 2010-09-28 2021-03-23 Amazon Technologies, Inc. Request routing information based on client IP groupings
US11025747B1 (en) 2018-12-12 2021-06-01 Amazon Technologies, Inc. Content request pattern-based routing system
US11075987B1 (en) 2017-06-12 2021-07-27 Amazon Technologies, Inc. Load estimating content delivery network
US11290418B2 (en) 2017-09-25 2022-03-29 Amazon Technologies, Inc. Hybrid content request routing system
US11604667B2 (en) 2011-04-27 2023-03-14 Amazon Technologies, Inc. Optimized deployment based upon customer locality

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100926362B1 (en) * 2007-08-27 2009-11-10 주식회사 케이티 Method and system for managing of server performance
KR100906440B1 (en) * 2008-05-07 2009-07-09 주식회사 엑셈 Apparatus for monitoring a performance of web applications by connecting was transaction with dbms session and method thereof
CN113190627A (en) * 2021-06-02 2021-07-30 南京恩瑞特实业有限公司 Nginx and Mycat based information system architecture and configuration method thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR19980070451A (en) * 1997-01-13 1998-10-26 디.볼레지니체크 Distributed Protocol Server
WO2000028464A2 (en) * 1998-11-05 2000-05-18 Bea Systems, Inc. A smart stub or enterprise javatm bean in a distributed processing system
WO2001098936A2 (en) * 2000-06-22 2001-12-27 Microsoft Corporation Distributed computing services platform
KR20020028963A (en) * 2002-03-04 2002-04-17 원동학 ADE data set component of client computer for transmitting/receiving of data between web servers

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR19980070451A (en) * 1997-01-13 1998-10-26 디.볼레지니체크 Distributed Protocol Server
WO2000028464A2 (en) * 1998-11-05 2000-05-18 Bea Systems, Inc. A smart stub or enterprise javatm bean in a distributed processing system
WO2001098936A2 (en) * 2000-06-22 2001-12-27 Microsoft Corporation Distributed computing services platform
KR20020028963A (en) * 2002-03-04 2002-04-17 원동학 ADE data set component of client computer for transmitting/receiving of data between web servers

Cited By (164)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7685272B2 (en) 2006-01-13 2010-03-23 Microsoft Corporation Application server external resource monitor
US9992303B2 (en) 2007-06-29 2018-06-05 Amazon Technologies, Inc. Request routing utilizing client location information
US10027582B2 (en) 2007-06-29 2018-07-17 Amazon Technologies, Inc. Updating routing information based on client location
US10305797B2 (en) 2008-03-31 2019-05-28 Amazon Technologies, Inc. Request routing based on class
US10511567B2 (en) 2008-03-31 2019-12-17 Amazon Technologies, Inc. Network resource identification
US11909639B2 (en) 2008-03-31 2024-02-20 Amazon Technologies, Inc. Request routing based on class
US11451472B2 (en) 2008-03-31 2022-09-20 Amazon Technologies, Inc. Request routing based on class
US9954934B2 (en) 2008-03-31 2018-04-24 Amazon Technologies, Inc. Content delivery reconciliation
US11245770B2 (en) 2008-03-31 2022-02-08 Amazon Technologies, Inc. Locality based content distribution
US10771552B2 (en) 2008-03-31 2020-09-08 Amazon Technologies, Inc. Content management
US9208097B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Cache optimization
US9210235B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Client side cache management
US9894168B2 (en) 2008-03-31 2018-02-13 Amazon Technologies, Inc. Locality based content distribution
US9887915B2 (en) 2008-03-31 2018-02-06 Amazon Technologies, Inc. Request routing based on class
US9544394B2 (en) 2008-03-31 2017-01-10 Amazon Technologies, Inc. Network resource identification
US9888089B2 (en) 2008-03-31 2018-02-06 Amazon Technologies, Inc. Client side cache management
US10158729B2 (en) 2008-03-31 2018-12-18 Amazon Technologies, Inc. Locality based content distribution
US11194719B2 (en) 2008-03-31 2021-12-07 Amazon Technologies, Inc. Cache optimization
US10530874B2 (en) 2008-03-31 2020-01-07 Amazon Technologies, Inc. Locality based content distribution
US9332078B2 (en) 2008-03-31 2016-05-03 Amazon Technologies, Inc. Locality based content distribution
US10797995B2 (en) 2008-03-31 2020-10-06 Amazon Technologies, Inc. Request routing based on class
US10554748B2 (en) 2008-03-31 2020-02-04 Amazon Technologies, Inc. Content management
US9407699B2 (en) 2008-03-31 2016-08-02 Amazon Technologies, Inc. Content management
US10157135B2 (en) 2008-03-31 2018-12-18 Amazon Technologies, Inc. Cache optimization
US10645149B2 (en) 2008-03-31 2020-05-05 Amazon Technologies, Inc. Content delivery reconciliation
US9479476B2 (en) 2008-03-31 2016-10-25 Amazon Technologies, Inc. Processing of DNS queries
US9621660B2 (en) 2008-03-31 2017-04-11 Amazon Technologies, Inc. Locality based content distribution
US9571389B2 (en) 2008-03-31 2017-02-14 Amazon Technologies, Inc. Request routing based on class
US9608957B2 (en) 2008-06-30 2017-03-28 Amazon Technologies, Inc. Request routing using network computing components
US9912740B2 (en) 2008-06-30 2018-03-06 Amazon Technologies, Inc. Latency measurement in resource requests
US9251112B2 (en) 2008-11-17 2016-02-02 Amazon Technologies, Inc. Managing content delivery network service providers
US11115500B2 (en) 2008-11-17 2021-09-07 Amazon Technologies, Inc. Request routing utilizing client location information
US9590946B2 (en) 2008-11-17 2017-03-07 Amazon Technologies, Inc. Managing content delivery network service providers
US11811657B2 (en) 2008-11-17 2023-11-07 Amazon Technologies, Inc. Updating routing information based on client location
US9985927B2 (en) 2008-11-17 2018-05-29 Amazon Technologies, Inc. Managing content delivery network service providers by a content broker
US10742550B2 (en) 2008-11-17 2020-08-11 Amazon Technologies, Inc. Updating routing information based on client location
US9451046B2 (en) 2008-11-17 2016-09-20 Amazon Technologies, Inc. Managing CDN registration by a storage provider
US11283715B2 (en) 2008-11-17 2022-03-22 Amazon Technologies, Inc. Updating routing information based on client location
US9734472B2 (en) 2008-11-17 2017-08-15 Amazon Technologies, Inc. Request routing utilizing cost information
US9515949B2 (en) 2008-11-17 2016-12-06 Amazon Technologies, Inc. Managing content delivery network service providers
US10116584B2 (en) 2008-11-17 2018-10-30 Amazon Technologies, Inc. Managing content delivery network service providers
US9444759B2 (en) 2008-11-17 2016-09-13 Amazon Technologies, Inc. Service provider registration by a content broker
US9787599B2 (en) 2008-11-17 2017-10-10 Amazon Technologies, Inc. Managing content delivery network service providers
US10523783B2 (en) 2008-11-17 2019-12-31 Amazon Technologies, Inc. Request routing utilizing client location information
US9191458B2 (en) 2009-03-27 2015-11-17 Amazon Technologies, Inc. Request routing using a popularity identifier at a DNS nameserver
US10601767B2 (en) 2009-03-27 2020-03-24 Amazon Technologies, Inc. DNS query processing based on application information
US10230819B2 (en) 2009-03-27 2019-03-12 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10264062B2 (en) 2009-03-27 2019-04-16 Amazon Technologies, Inc. Request routing using a popularity identifier to identify a cache component
US9237114B2 (en) 2009-03-27 2016-01-12 Amazon Technologies, Inc. Managing resources in resource cache components
US10491534B2 (en) 2009-03-27 2019-11-26 Amazon Technologies, Inc. Managing resources and entries in tracking information in resource cache components
US10574787B2 (en) 2009-03-27 2020-02-25 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10521348B2 (en) 2009-06-16 2019-12-31 Amazon Technologies, Inc. Managing resources using resource expiration data
US10783077B2 (en) 2009-06-16 2020-09-22 Amazon Technologies, Inc. Managing resources using resource expiration data
US10135620B2 (en) 2009-09-04 2018-11-20 Amazon Technologis, Inc. Managing secure content in a content delivery network
US9130756B2 (en) 2009-09-04 2015-09-08 Amazon Technologies, Inc. Managing secure content in a content delivery network
US10785037B2 (en) 2009-09-04 2020-09-22 Amazon Technologies, Inc. Managing secure content in a content delivery network
US9712325B2 (en) 2009-09-04 2017-07-18 Amazon Technologies, Inc. Managing secure content in a content delivery network
US10218584B2 (en) 2009-10-02 2019-02-26 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9246776B2 (en) 2009-10-02 2016-01-26 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9893957B2 (en) 2009-10-02 2018-02-13 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9240954B1 (en) 2009-10-02 2016-01-19 Amazon Technologies, Inc. Forward-based resource delivery network
US8433771B1 (en) * 2009-10-02 2013-04-30 Amazon Technologies, Inc. Distribution network with forward resource propagation
US11205037B2 (en) 2010-01-28 2021-12-21 Amazon Technologies, Inc. Content distribution network
US10506029B2 (en) 2010-01-28 2019-12-10 Amazon Technologies, Inc. Content distribution network
US9495338B1 (en) 2010-01-28 2016-11-15 Amazon Technologies, Inc. Content distribution network
US9009676B2 (en) 2010-04-02 2015-04-14 International Business Machines Corporation Common recording and replaying performance test tool for a client-server environment
US9787775B1 (en) 2010-09-28 2017-10-10 Amazon Technologies, Inc. Point of presence management in request routing
US9185012B2 (en) 2010-09-28 2015-11-10 Amazon Technologies, Inc. Latency measurement in resource requests
US9712484B1 (en) 2010-09-28 2017-07-18 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US10958501B1 (en) 2010-09-28 2021-03-23 Amazon Technologies, Inc. Request routing information based on client IP groupings
US10079742B1 (en) 2010-09-28 2018-09-18 Amazon Technologies, Inc. Latency measurement in resource requests
US9253065B2 (en) 2010-09-28 2016-02-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9794216B2 (en) 2010-09-28 2017-10-17 Amazon Technologies, Inc. Request routing in a networked environment
US10097398B1 (en) 2010-09-28 2018-10-09 Amazon Technologies, Inc. Point of presence management in request routing
US9191338B2 (en) 2010-09-28 2015-11-17 Amazon Technologies, Inc. Request routing in a networked environment
US9497259B1 (en) 2010-09-28 2016-11-15 Amazon Technologies, Inc. Point of presence management in request routing
US10931738B2 (en) 2010-09-28 2021-02-23 Amazon Technologies, Inc. Point of presence management in request routing
US9407681B1 (en) 2010-09-28 2016-08-02 Amazon Technologies, Inc. Latency measurement in resource requests
US10015237B2 (en) 2010-09-28 2018-07-03 Amazon Technologies, Inc. Point of presence management in request routing
US11108729B2 (en) 2010-09-28 2021-08-31 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US9160703B2 (en) 2010-09-28 2015-10-13 Amazon Technologies, Inc. Request routing management based on network components
US10778554B2 (en) 2010-09-28 2020-09-15 Amazon Technologies, Inc. Latency measurement in resource requests
US10225322B2 (en) 2010-09-28 2019-03-05 Amazon Technologies, Inc. Point of presence management in request routing
US9800539B2 (en) 2010-09-28 2017-10-24 Amazon Technologies, Inc. Request routing management based on network components
US11336712B2 (en) 2010-09-28 2022-05-17 Amazon Technologies, Inc. Point of presence management in request routing
US9930131B2 (en) 2010-11-22 2018-03-27 Amazon Technologies, Inc. Request routing processing
US10200492B2 (en) 2010-11-22 2019-02-05 Amazon Technologies, Inc. Request routing processing
US10951725B2 (en) 2010-11-22 2021-03-16 Amazon Technologies, Inc. Request routing processing
US9391949B1 (en) 2010-12-03 2016-07-12 Amazon Technologies, Inc. Request routing processing
US11604667B2 (en) 2011-04-27 2023-03-14 Amazon Technologies, Inc. Optimized deployment based upon customer locality
US9628554B2 (en) 2012-02-10 2017-04-18 Amazon Technologies, Inc. Dynamic content delivery
US10021179B1 (en) 2012-02-21 2018-07-10 Amazon Technologies, Inc. Local resource delivery network
US10623408B1 (en) 2012-04-02 2020-04-14 Amazon Technologies, Inc. Context sensitive object management
US10225362B2 (en) 2012-06-11 2019-03-05 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US11303717B2 (en) 2012-06-11 2022-04-12 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US11729294B2 (en) 2012-06-11 2023-08-15 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9154551B1 (en) 2012-06-11 2015-10-06 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9525659B1 (en) 2012-09-04 2016-12-20 Amazon Technologies, Inc. Request routing utilizing point of presence load information
US10015241B2 (en) 2012-09-20 2018-07-03 Amazon Technologies, Inc. Automated profiling of resource usage
US9323577B2 (en) 2012-09-20 2016-04-26 Amazon Technologies, Inc. Automated profiling of resource usage
US10542079B2 (en) 2012-09-20 2020-01-21 Amazon Technologies, Inc. Automated profiling of resource usage
US10645056B2 (en) 2012-12-19 2020-05-05 Amazon Technologies, Inc. Source-dependent address resolution
US10205698B1 (en) 2012-12-19 2019-02-12 Amazon Technologies, Inc. Source-dependent address resolution
US9294391B1 (en) 2013-06-04 2016-03-22 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US10374955B2 (en) 2013-06-04 2019-08-06 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9929959B2 (en) 2013-06-04 2018-03-27 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9792365B2 (en) 2013-12-31 2017-10-17 Clicktale Ltd. Method and system for tracking and gathering multivariate testing data
WO2015103122A3 (en) * 2013-12-31 2015-08-20 Clicktale Ltd. A method and system for tracking and gathering multivariate testing data
US10091096B1 (en) 2014-12-18 2018-10-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US11381487B2 (en) 2014-12-18 2022-07-05 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10097448B1 (en) 2014-12-18 2018-10-09 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US11863417B2 (en) 2014-12-18 2024-01-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10728133B2 (en) 2014-12-18 2020-07-28 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10033627B1 (en) 2014-12-18 2018-07-24 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10225326B1 (en) 2015-03-23 2019-03-05 Amazon Technologies, Inc. Point of presence based data uploading
US11297140B2 (en) 2015-03-23 2022-04-05 Amazon Technologies, Inc. Point of presence based data uploading
US9887932B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US9819567B1 (en) 2015-03-30 2017-11-14 Amazon Technologies, Inc. Traffic surge management for points of presence
US10469355B2 (en) 2015-03-30 2019-11-05 Amazon Technologies, Inc. Traffic surge management for points of presence
US9887931B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US10691752B2 (en) 2015-05-13 2020-06-23 Amazon Technologies, Inc. Routing based request correlation
US10180993B2 (en) 2015-05-13 2019-01-15 Amazon Technologies, Inc. Routing based request correlation
US11461402B2 (en) 2015-05-13 2022-10-04 Amazon Technologies, Inc. Routing based request correlation
US9832141B1 (en) 2015-05-13 2017-11-28 Amazon Technologies, Inc. Routing based request correlation
US10616179B1 (en) 2015-06-25 2020-04-07 Amazon Technologies, Inc. Selective routing of domain name system (DNS) requests
US10097566B1 (en) 2015-07-31 2018-10-09 Amazon Technologies, Inc. Identifying targets of network attacks
US9742795B1 (en) 2015-09-24 2017-08-22 Amazon Technologies, Inc. Mitigating network attacks
US9794281B1 (en) 2015-09-24 2017-10-17 Amazon Technologies, Inc. Identifying sources of network attacks
US9774619B1 (en) 2015-09-24 2017-09-26 Amazon Technologies, Inc. Mitigating network attacks
US10200402B2 (en) 2015-09-24 2019-02-05 Amazon Technologies, Inc. Mitigating network attacks
US10270878B1 (en) 2015-11-10 2019-04-23 Amazon Technologies, Inc. Routing for origin-facing points of presence
US11134134B2 (en) 2015-11-10 2021-09-28 Amazon Technologies, Inc. Routing for origin-facing points of presence
US10257307B1 (en) 2015-12-11 2019-04-09 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10049051B1 (en) 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10348639B2 (en) 2015-12-18 2019-07-09 Amazon Technologies, Inc. Use of virtual endpoints to improve data transmission rates
US11463550B2 (en) 2016-06-06 2022-10-04 Amazon Technologies, Inc. Request management for hierarchical cache
US10075551B1 (en) 2016-06-06 2018-09-11 Amazon Technologies, Inc. Request management for hierarchical cache
US10666756B2 (en) 2016-06-06 2020-05-26 Amazon Technologies, Inc. Request management for hierarchical cache
US11457088B2 (en) 2016-06-29 2022-09-27 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US10110694B1 (en) 2016-06-29 2018-10-23 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US10516590B2 (en) 2016-08-23 2019-12-24 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US9992086B1 (en) 2016-08-23 2018-06-05 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US10033691B1 (en) 2016-08-24 2018-07-24 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US10469442B2 (en) 2016-08-24 2019-11-05 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US10469513B2 (en) 2016-10-05 2019-11-05 Amazon Technologies, Inc. Encrypted network addresses
US10616250B2 (en) 2016-10-05 2020-04-07 Amazon Technologies, Inc. Network addresses with encoded DNS-level information
US11330008B2 (en) 2016-10-05 2022-05-10 Amazon Technologies, Inc. Network addresses with encoded DNS-level information
US10505961B2 (en) 2016-10-05 2019-12-10 Amazon Technologies, Inc. Digitally signed network address
US10372499B1 (en) 2016-12-27 2019-08-06 Amazon Technologies, Inc. Efficient region selection system for executing request-driven code
US11762703B2 (en) 2016-12-27 2023-09-19 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10831549B1 (en) 2016-12-27 2020-11-10 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10938884B1 (en) 2017-01-30 2021-03-02 Amazon Technologies, Inc. Origin server cloaking using virtual private cloud network environments
US10503613B1 (en) 2017-04-21 2019-12-10 Amazon Technologies, Inc. Efficient serving of resources during server unavailability
US11075987B1 (en) 2017-06-12 2021-07-27 Amazon Technologies, Inc. Load estimating content delivery network
US10447648B2 (en) 2017-06-19 2019-10-15 Amazon Technologies, Inc. Assignment of a POP to a DNS resolver based on volume of communications over a link between client devices and the POP
US11290418B2 (en) 2017-09-25 2022-03-29 Amazon Technologies, Inc. Hybrid content request routing system
US10592578B1 (en) 2018-03-07 2020-03-17 Amazon Technologies, Inc. Predictive content push-enabled content delivery network
US10862852B1 (en) 2018-11-16 2020-12-08 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US11362986B2 (en) 2018-11-16 2022-06-14 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US11025747B1 (en) 2018-12-12 2021-06-01 Amazon Technologies, Inc. Content request pattern-based routing system
US20210056004A1 (en) * 2019-08-22 2021-02-25 Micro Focus Llc Regression testing of computer systems using recorded prior computer system communications
US11579993B2 (en) * 2019-08-22 2023-02-14 Micro Focus Llc Regression testing of computer systems using recorded prior computer system communications
CN111324545B (en) * 2020-03-13 2023-09-22 中国工商银行股份有限公司 Application server middleware performance test method, test terminal and application server
CN111324545A (en) * 2020-03-13 2020-06-23 中国工商银行股份有限公司 Application server middleware performance test method, test terminal and application server

Also Published As

Publication number Publication date
KR100447041B1 (en) 2004-09-07
KR20040013058A (en) 2004-02-11

Similar Documents

Publication Publication Date Title
WO2005071560A1 (en) Method and system for application performance management
EP1386240B1 (en) Synthetic transaction monitor
EP1490775B1 (en) Java application response time analyzer
US9727405B2 (en) Problem determination in distributed enterprise applications
US6505248B1 (en) Method and system for monitoring and dynamically reporting a status of a remote server
US6167448A (en) Management event notification system using event notification messages written using a markup language
US6983317B1 (en) Enterprise management system
US7792948B2 (en) Method and system for collecting, aggregating and viewing performance data on a site-wide basis
US7181519B2 (en) Distributed network monitoring and control system
US6205413B1 (en) End-user oriented performance monitoring system for interactive end-to-end data communications
US7680907B2 (en) Method and system for identifying and conducting inventory of computer assets on a network
US20070192150A1 (en) Method and apparatus for identifying problem causes in a multi-node system
US20050060425A1 (en) Application-based autonomic connectivity
US20020002581A1 (en) Messaging based proxy application management
US20020194320A1 (en) Remote support system
US20080162690A1 (en) Application Management System
US8326913B2 (en) Method and system for service contract discovery
Kaushik Use of open source technologies for enterprise server monitoring using SNMP
US20020124087A1 (en) System and method for providing dynamic service using front-end
Terplan 3.11 Performance Management of Intranets

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase