Сообщения

Сообщения за ноябрь, 2017

Overview of the most important features of Postgres 9.3: materialized views

PostgreSQL 9.3 will come out with a pretty cool feature called materialized views . Feature was developed by Kevin Grittner and not so long ago sakimichan: the commit 3bf3ab8c563699138be02f9dc305b7b77a724307 Date: Sunday March 4, 18:23:31 2013 -0600 Author: Kevin Grittner Added materialized views Have a materialized view has a rule, as well as the regular performances, and a lot of, as well as other physical properties like a table. The rule is used only for filling of the table references in the query point to the materialized data. Implemented minimal functionality, but it can be useful in many cases. Currently, data is loaded only “on demand” CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW. It is expected that future releases will be added incrementally update the data with different settings time updates, and will be given clearer definition to the concept of “fresh” data. At some point, even the queries to use the materialized data instead of the dat

Comparing the performance of MongoDB vs. PostgreSQL. Part II: Index

Изображение
Continued from here . Experiment II: Index For this experiment, we created indexes on the fields id and floatvalue (text fields lowered, the topic full-text index will not affect as it is material for another article). As the queries used samples from the ranges: the the 10 000 < id < 100 000 200 000 < floatvalue < 300 000 But first, you must assess how much speed fell inserting after adding of indexes. For this, we will add another 250 000 records in MongoDB and POstgreSQL. MongoDB the Insert 250000 records complete! Total time: 69.453 sec PostgreSQL the psql -d prefTest -f 250k.p5.sql (Total time: 466.153 sec) After simple calculations, we can understand that the speed of insert of MongoDB remained the undisputed leader: after adding indexes its insertion speed fell by only ~10% and made 3600 items per second . While the speed of inserts from PostgreSQL fell on ~30% was about 536 records per second . I would like

Optimizing queries. The basics of EXPLAIN in PostgreSQL (part 2)

Изображение
Podolzhayut to publish the author's processing Understanding EXPLAIN from Guillaume Lelarge. Again note that some information has been omitted for brevity, so I strongly recommend to read the original. Previous part: Part 1 Cache What happens on a physical level, with follow through with our request? Face it. My server raised to Ubuntu 13.10. Use disk level caches of the OS. Stop PostgreSQL, force detecting changes in the file system, clear the caches, start PostgreSQL: the > sudo service postgresql-9.3 stop > sudo sync > sudo su - # echo 3 > /proc/sys/vm/drop_caches # exit > sudo service postgresql-9.3 start Now the caches are cleared, try to execute the query with the option BUFFERS the EXPLAIN (ANALYZE,BUFFERS) SELECT * FROM foo; QUERY PLAN — Seq Scan on foo (cost=0.00..18334.10 rows=1000010 width=37) (actual time=0.525..734.754 rows=1000010 loops=1) Buffers: shared read=8334 Total runtime: 1253.177 ms (3 rows) The table is

Search using Lucene in Playframework 1.x

In my web project on Playframework-e at one point, it took the search. The idea of a database search using like I'm immediately dismissed because I like ranking and other buns "smart" search, and to invent a Bicycle was neither the time nor the desire. As the project is in Java — it was very tempting to use Lucene for this. In Google I found a great module for Playframework-called Search , was also found module Elastic Search , which also uses Lucene, but it requires installing a separate server, and therefore was a tad off. Module Search liked its simplicity — all the "bells and whistles" in it is encapsulated, so using them is very easy. With the installation of the module, as always in Play-e, no problems, the command play install search worked on "cheers" and took out a module from a repository. Adding module.search=${play.path}/modules/search-2.0 application.conf I was able to use it in the application. Following a brief leadership,

PgTune — tuning PostgreSQL performance for a given hardware configuration (online version)

Изображение
Hello, habrovany. Today I want to talk about such interesting things as PgTune. In 2008 Gregory Smith has created a good tool pgtune . The main objective of this tool was to help newcomers with setting up PostgreSQL. Utility is great, but has a small cons: the the Not updated since 2009 (so the calculations a bit is no longer relevant steel) the Need to download or install from the packages (not a problem in many Linux distributions it is) So I have created an online version of PgTune . Main advantages: the the Updated count to configuration. the you do Not need to put anything or to pump the Works in offline mode, and it can be used as mobile app On first boot You will no longer need access to the Internet (only for updates) to use the utility. This is implemented using AppCache technology. That would put it as an app on the iPhone, just log in iOS Safari or Android Chrome and select the menu item "Add to desktop". And mos

Restore PostgreSQL database from WAL-backup skip parts of entries

introduction In PostgreSQL, there is an interesting technical solution — before actually start to change something in the files of the database itself, the DBMS writes is translated into an internal format command in a special log — Write-Ahead Log, and after successful completion of the transaction, makes the journal a note. This was done for disaster recovery, but in the end the inquisitive mind of the developers came to the idea to use this log for backup and replication. In principle, logically, all the moves it contains, moreover can not simply restore the data from backup and recover the database state at a specific point in time, interrupting the playback of the records WAL-log at the right time. However, let's consider this scenario — let's say on Monday you did a basic backup and started the archiving of WAL logs, in the environment you made the request to remove the erroneous mask, and only found out about it on Friday, when the Manager reported the disappeara

Service time zones in Ruby and Postgis

Изображение
In one of the projects in which I participated, arose the problem of determining the time zone for the current geolocation of the user. On the backend came a record generated by the user through the smartphone. The time has come not in UTC, but the provided coordinates. Of course, there are services (like The Google Time Zone), but they all paid or are severely limited in functionality. So I decided to write my own. The service should be as simple as possible. We need to make only one request of the form the http://host/timezone/name?lat=55.2341&lng=43.23352 Where lat is the latitude, lng — longitude. Nastraivaet database As the database will use PostgreSQL . We will also need extension Postgis custom-tailored to work with geographic objects. We assume that PostgreSQL is already installed. If not, the Internet has a lot of guides and tutorials on how to do it. The process of installing Postgis also should not cause difficulties on the official website has

"Rostelecom" will start the state Internet search engine

Изображение
Rostelecom is developing a new search engine "Sputnik", which could be launched in 2014. As told Vedomosti's interlocutor in "Rostelecom", the search engine, which will be located at sputnik.ru to begin operation in the first quarter of 2014. He noted that the project exists about three years, and its active development is done in the last year or two, for which the company invested $ 20 million. According to the newspaper, the new searcher will move "at the state level," and by default be installed in government and state-owned companies. However, a source in Rostelecom said that the search engine will provide a filtration objectionable to the state content and for its promotion among users will use market methods. System development deals with the company "KM Media", acquired Rostelecom in 2012 (the link to Roem.ru interesting comments, read). The newspaper's sources in "Yandex" and Mail.ru Group said t

Developing modules for MODX Evolution in 2017 for the little ones

Изображение
What are modules In MODX EVO presents the following types of resources, templates, chunks, snippets, plugins and modules. If you work in the system that know exactly what answer the first three, but perhaps personally never worked with the latter. The plugin is integrated into the MODX code which allows you to change the mechanics of interaction with the admin interface with plugins you can change the way data is displayed for editing of any resource, and ask how and at what stages of interaction should make. Simply put with the help of plugins you hang your handlers. But in this article I want to talk a little bit about the other, about the modules. This article I'm writing for the reason that the Internet is very little information on this subject, in the Russian segment, I do not found. a Module is an application based on the architecture of MODX and extend the capabilities of the control system. The module may group the set of elements (snippets, chunks, data), impl

Popular about Amazon Web Services

Изображение
Introduction This report will be considered by the AWS, that uses our platform and with which I am familiar not by hearsay. I'm working on a project that utilizes almost all the possible services, and we aim in the near future to cover even more possibilities offered to us by Amazon. Management of AWS is how to use the web interface (AWS console), and use Command Line Tools. The console contains all the AWS services, but the functionality of multiple cropped. At the command prompt also allows you more flexibility to configure a particular service, are also available in closed console functions. Amazon Elastic Compute Cloud (EC2) Description EC2 is a cloud — based service that provides virtual servers (Amazon EC2 Instance), 2 types of data stores, as well as a load balancer (Load Balancer). Many of you are familiar with VPS — Virtual Private Server. Now, EC2 is not that other, as a service that provides VPS in the present cloud where the server can easily migra

Typography habré

Изображение
You should, when writing hepatophyta, add the option to automatically replace the correct quotation marks on wrong and dashes to hyphens? That is to add this printer , about which you know so well. It would look something like this: Of course, you say, if someone has the desire, he can go to the same website or to install a program that when you press the magic buttons all "tipographic". Yes, it can. Maybe even a program to write. But first, why somewhere to go or something to write and secondly if this option is right in Habra, more people will use it. What a gut, so as to read tipografiya text much more pleasant. What I propose to use a slightly simplified version of the beetle, that is, to use just replace the quotes, dashes, and the conclusion what should the nobr tags (except that the tags code and pre). And no plagiarism. Program write is not a problem even without looking at same source and the idea is as old as the world. The program works fine a

Comparing Windows Azure Table Storage and Amazon DynamoDB

Hello. I offer a translation of the first article of the cycle comparison of the services provided by Windows Azure and Amazon that is quite famous in the circles of the cloud specialist — Gaurav Mantri. In this article, I compare Windows Azure Table Storage and Amazon DynamoDB – WATS and ADDB respectively. From the point of view of functionality WATS and ADDB provide similar opportunities. Both of NoSQL systems designed to store large amounts of data. Amazon also has another database NoSQL SimpleDB . An important point to be noted is the fact that ADDB is not just a NoSQL database. This database service. Yes, it is true that it is used to manage data, but you control the scalability of the system using the bandwidth that you need. In this sense this is very similar to instances of the compute service Amazon or Windows Azure. In the case of the instance of the compute service you choose what instance size you need and the system responds to the request. Similarly in