5 главных алгоритмов сэмплинга / Хабр

(pdf) sample preparation for the determination of metals in food samples using spectroanalytical methods – a review

0 Numbering systems: Serial and three-dimensional lot numbering

A numbering system for the lot must be decided upon before obtaining the random numbers for the sample. Two methods of numbering a lot are “serial lot numbering”, and “three-dimensional numbering”.

0 Purpose

The purpose of this document is to provide Canadian Food Inspection Agency (CFIA) inspection staff with the general procedures for sampling food and food production environments, as well as water and ice that come into contact with food. This guidance is to ensure that such samples are representative of the food and the food production environment.

This document is intended to be used in conjunction with the Standard Inspection Procedures (SIP) and the Operational Guidelines for specific food commodities.

The guidance outlined below may be used when verifying compliance of a food product, to aid in assessment of a Preventive Control Plan (PCP) related sub-element, as part of a food safety investigation, or follow-up to a complaint.

Sampling frequencies and sample unit details are provided in the CFIA Sampling Information.

1 Serial lot numbering

In a simple arrangement, such as packages on a shelf or on a packing line, the packages in the lot can be considered to be numbered from 1 to n, where “n” is the total number of packages. The inspector does not have to mark the packages with numbers, but each package should have a number assigned to it by memory.

If the packages are in only one layer, they may be found or arranged in rows and columns on the shelf. Imagine that the packages are numbered from 1 to 15 in some systematic fashion, as illustrated below:

If there is more than one layer of packages, extend the serial numbering system layer by layer. In this example, the second layer would be considered to be packages numbered 16 through 30, the third layer, packages numbered 31 through 45, the fourth numbered 46 through 60, and so on, as illustrated below:

2 Three-dimensional lot numbering

When a large stack of packages or cases must be numbered, the inspector can use a three-dimensional lot numbering system using the Right, Up and to the Back method, referred to as “RUB”.

0 Authorities

The inspection powers, including the authorization to take samples, by the above legislation are identified and explained in the Operational guideline – Food regulatory response guidelines.

0 Acronyms

Acronyms are spelled out the first time they are used in this document and are consolidated in the Food business line acronyms list.

1 General principles

1. Follow principles of good personal hygiene; inspectors must not be a source of contamination:
   1. Keep hands, fingernails, clothing, and shoes/boots thoroughly clean at each visit
   2. Removed personal effects and jewellery prior to collecting a sample
   3. Secure effects that cannot be removed (under clothes or gloves)
2. Conduct sampling to maintain the integrity and continuity of the sample associated with the lot, from the time the sample is drawn to the completion of the inspection.
3. Collect samples when the lot can be accessed without interference. If there are conditions that cause interference with sampling, document the issue and report the issue to the Inspection Supervisor.
4. Begin the sampling sequence in the finished product area first (least contaminated) and proceed through the potentially contaminated areas, finishing at the raw materials and receiving areas.
5. Conduct sample collection aseptically to avoid contaminating the sample, the product, the environment and the inspector.
6. Label each sample unit with adequate information so that the unit can be matched to the sampling submission form. The markings must be legible and permanent.
7. Employ a sample identification system that permits an inspector to:
   1. assign a unique identification number for the sample associated with a lot
   2. affix all pertinent information to the sample, and
   3. document all sampling information for record keeping purposes

2 Types of sampling plans

There are different types of sampling plans based on the design and the purpose of the sampling and testing, including monitoring, directed, compliance, and blitzes or special surveys. These terms are described in the Canadian Food Inspection Agency – Health Canada Food Sampling and Testing Terminology.

Legal sampling is a sanctioned method in which continuity and chain of custody are maintained. It is undertaken for specific conditions where legal action is the anticipated follow up action. Certain additional criteria are demanded during the sampling submission and laboratory testing of these samples.

3.1 Prepare for sampling

Prior to collecting samples, perform the following in-office steps:

1. Plan sampling activities, whenever possible, so that samples are not stored or shipped over the weekend. In some cases, the laboratory will accept samples for analysis on the weekend. Contact the laboratory in advance to determine availability.
2. Determine the intent of the sampling
   1. To support inspection findings,
   2. To fulfill the CFIA Annual Sampling Plan,
   3. To assess human exposure to food-related risks, etc.
3. Prepare the Inspector toolkit and additional materials or protective equipment required. Sampling equipment/materials are selected as appropriate for the collection, preparation, storage and transportation samples and for maintaining the condition of the samples. Refer to Inspector toolkit (internal access only – RDIMS 11289973) for the list of equipment that may be required for sampling. If visiting multiple facilities in one day, respect biosecurity and obtain sufficient supplies for each site. Follow appropriate biosecurity measures for preparation, cleaning and sanitizing equipment, prior to arriving at the regulated party’s premises. (Refer to the SIP Appendix I for Biosecurity)
4. Acquire or assign an appropriate series of laboratory sample numbers to be used to uniquely identify the samples. Refer to the CFIA Sampling Information (internal access only) for guidance on lab sample numbers and how to proceed if additional samples per sampling plan are required and for special request sampling procedures.

3.2 Determine which food products to sample

Choose a product to sample by considering the following factors:

• Product meets the sampling plan criteria (for example milk with added vitamins)
• Compliance history of the product, country, establishment (licenced establishment, importer or exporter)
• Lot size
• Availability of the lot
• Date the product was last sampled at the premises
• Any other circumstance the inspector may be aware of

3.3 Locate and identify the lot

Lots are typically differentiated by logical breaks in production (such as different source of honey), but should be limited to a single production day or less.

Each lot should be prepared for sampling in such a way that the sample(s) can be taken without hindrance. If a portion of the lot has already been distributed, the remaining quantity of the lot is to be considered the sampling lot.

Obtain the following information, where applicable:

• Lot size (number of cases, number of containers per case)
• Lot codes and their interpretation
• Brand name
• Product type and style of pack
• Container type and unit weight
• Name and address of agent/owner
• Country of origin or destination

3.4 Define the sample unit

Sample Units may consist of:

• A whole fruit, vegetable or natural bunch (such as grapes)
• A whole animal or complete animal part or organ
• The smallest discrete package or where the smallest packages are very small, a pack of packages may form the unit
• Smaller portions taken from bulk materials and large packages (such as drums, cheeses, etc.) which are individually too large to be taken as samples

When a lot consists of pre-packaged product, each package constitutes a sample unit.

If the container sizes available for sampling differ from the sample size required, adjust the number of sample units accordingly. For example, if the required sample size is 1 unit (500 g) but only 250 g containers are available, two containers will be required from the same lot to meet the 500 g sample size requirement.

Conversely, if only larger containers are available, the entire container can be sampled or a 500 g sample can be drawn. If the required sample size requires a number of sub samples, for example 6 units × 250 g, 6 units of different containers must be taken regardless of container size.

3.5 Sample collection

• Collect samples aseptically
  Accurate and reliable analytical results depend on proper sampling. If the samples are not obtained in a manner that ensures and protects the integrity of the sample, the results may become irrelevant. If the units to be sampled are in an unclean area of the establishment, it may be necessary to take the units and move to a cleaner area to obtain the samples.
  For aseptic sampling, use pre-sterilized plastic or metal tools. If pre-sterilized plastic or metal tools are unavailable, metal tools can be sterilized with 70% alcohol immediately before use. The metal tool should be dry before being used to collect samples.
  If it is necessary to drill, saw, or cut the item being sampled (such as large frozen fish, cheese wheels, frozen fruit, etc.), use pre-sterilized stainless steel bits, blades, knives, etc. Sampling instruments with wooden handles are particularly susceptible to bacterial contamination and are difficult to sterilize, they should be avoided.
• Collect representative samples (Random sampling)
  In order to randomly select samples, each unit (container or package) in the lot must have an equal chance of being selected, thereby excluding bias. Select the samples as randomly as possible within the lot, given the constraints of working in a production facility or warehouse environment. A true random selection of pallets, cases and/or individual packages can be achieved by using a random sampling technique. A numbering system in combination with a random number generator is one of several methods that can be used to ensure randomness. Refer to Appendix 7 for Random sampling procedure
• Sampling for multiple analyses
  When a sample unit is drawn for more than one analysis, ensure the sample unit is of sufficient mass to perform all of the required analyses. If microbiological analysis is one of the required analyses, submit the samples to the microbiological section for analysis first to ensure the integrity of the sample is not jeopardized.

3.6 Identify and document the sample unit

Record the details of sampling, for example, lot location, no. of samples drawn, unique identification number, time of sampling, codes drawn

Ensure all samples are accompanied by a completed food product sampling submission form in the LSTS application. Include the following information if available and where appropriate:

• Type of analysis required (sulphite, net weight, etc.)
• Packer and packer code
• Shipment identification number
• Held tag number (if product is detained)
• Lot size and unit weight
• Sampler’s name
• Lake code (body of water and landmarks), statistical area and sub-area
• Length and weight of fish (contaminant sampling)
• Number of units sampled
• Plant name and registration number
• Harvest site (shellfish samples)
• Harvest date (shellfish samples)
• Country of origin
• Collection date and time
• Grower/field crop ID (fresh fruit products)
• Processing date
• Species and product type
• Farm and pen information (farmed fish)
• Inspection status and type (alert, random, etc.)
• Name of importer
• Analyses required for export certificate
• Cost recoverable (yes/no)
• Producer premises code (shell egg labelling requirement)

Include any other relevant information which would assist in performing the analysis or assessing the results, such as:

• A copy of the label
• Observations of abnormal odours, taste, colours, or texture

4 Environmental sampling

Environmental sampling for the detection of microorganisms is employed to determine the presence of pathogens or other organisms on equipment and food contact surfaces. The detection of microorganisms may indicate poor sanitation or microbial growth niches in a food establishment. Environmental sampling consists of Food Contact Surface (FCS) and Non Food Contact Surface (NFCS) sampling.

The sampling technique chosen will depend on type of area to be sampled. For example, swabbing is better for material sampling, while exposure of agar is well suited for ambient air sampling. Details on different sampling techniques are found in Appendix 6.

4.1 Prepare for sampling

Take the following steps prior to sampling:

1. Obtain a copy of the establishment’s environmental sampling program
2. Acquire a process flow chart, including the employee and product traffic patterns
3. Identify sampling sites and target microorganisms or hazards. Sampling sites may include walls, floors, drains, equipment, food contact surfaces, etc.
4. To assess cleaning and sanitation conditions in the facility, plan to take environmental samples immediately after cleaning and sanitation, but before production starts
5. To assess microbiological conditions during production, plan to take samples during production, typically 3 hours after start of operation or at the mid-time production when production is less than 3 hours
6. To assess the adequacy of the hazard analysis
   • for lines producing ready-to-eat (RTE) products subjected to a heat process treatment, plan sampling to start at the post-lethality step
   • for lines producing RTE products not subject to a heat treatment process, plan to take samples at the step where the product is in its final RTE form. For example, for cold-smoked fish, sampling should start at the cold smoking step; for sushi, sampling should start at the step where all the ingredients are assembled as sushi.
7. Identify the line(s) with higher risk and select the collection sites (food contact surfaces only) in the post-lethality treatment areas of the establishment with the highest probability of contamination.
8. Assemble the Inspection Toolkit

4.1.1 Prepare environmental sampling kits (as required)

1. Assemble the Inspection Toolkit for environmental sampling:

* Note – New CFIA swabs can be ordered by following this guidance: “Environmental Swabbing Kit and Ordering Process” (internal access only – RDIMS 3413640)

** Note – non-sterile gloves may be sterilized by spraying with a 70% Ethanol spray or appropriate sanitizer and allowed to air dry.

4.3 Dried or accumulated material sampling

In addition to surface sampling, there may be evidence of dried or accumulated materials on processing lines. The sample size of such material will be determined by the analysis required and/or the amount of sample available.

5 Water and ice sampling

Water samples are taken in order to evaluate the quality of the water source and the characteristics of water used as an ingredient and/or in treatments/cleaning.

Quantity and size of the unit sample are indicated in the sampling plan and are established according to the type of analysis done in laboratory.

5.1 General principles related to water sampling

• Wash and dry hands before sampling
• Take sample aseptically
• Use sterile gloves if needed (for example, when sampling water for microbiological analysis)
• Use sterile container and do not rinse it before taking the sample
• When sampling water treated with chlorine, use a sample container with sodium thiosulfate to inactivate chlorine

5.2 Running water source

Collect the sample from a source water outlet within the facility. Do not use a hydrant, hose or any faucet located outside of building. Do not sample from a tap that is obviously contaminated.

1. Request the establishment to remove any aerators or other devices from the outlet to be sampled. These devices may be heavily contaminated and may alter the water sample result
2. Disinfect the end of the faucet with alcohol wipes
3. Run the cold water for at least two to three minutes
4. Adjust the flow of the stream to ensure that no splashing occurs
5. Fill the container to the fill line or within 6-7 mm (1/4 inch) of the top. The sampling container should not be overfilled. If the sampling container is accidentally overfilled, discard the container and use a new sampling container.

5.3 Ice

A sample, consisting of ten sample units drawn at random from each lot, shall be taken. Each sample unit shall consist of at least 100 ml or g. Collect original unopened container wherever possible. Place each collected sample unit into a separate sterile container. Do not allow sample units of prepackaged ice to thaw during shipment.

6 Transportation and storage

Samples should be kept in such a manner that the controlled characteristic is not modified. For example, use of a sterile and cooled container is mandatory for samples that will be analysed for microbiological control.

Transport samples in clean and dry containers offering adequate protection from external contamination and protection against damage to the sample in transit. Refrigerated or frozen samples should be transported in insulated containers with a sufficient quantity of ice packs or dry ice to maintain sample integrity and the same conditions at which they were sampled.

If samples are not shipped immediately, store them in a secure location in the office. Keep frozen samples in a freezer (-18°C), refrigerated samples at an appropriate temperature (0 to 4 °C) and products which do not require refrigeration storage conditions at ambient room temperature.

8.1 Samples judged Investigative (I) or Unsatisfactory (U)

When a sample is judged investigative or unsatisfactory, evaluate the results and determine if there is a violation and/or a potential health and safety concern with respect to product on the market. Consult the Standard Regulatory Response Procedure, and any Food or Commodity specific guidance to help identify any regulatory response options and strategies.

If additional guidance is required, consult other personnel such as the Recall Coordinator, the Inspection Supervisor or Inspection Manager (IM) or designate, and others as appropriate for resolution of the issue. The degree of consultation may vary depending upon the issue, from notification by e-mail to conference calls and meetings.

If product has left control of the regulated party, a food safety investigation may be required to ensure all aspects of the potential health and safety issue are addressed. Follow the guidance in the Food Safety Investigation Response.

When a potential health and safety issue is identified, initiate the regulatory response process to limit the exposure to the potential health risk or to prevent further violation(s) of the SFCA and SFCR, the FDA and the FDR.

When a potential health and safety issue is not identified or a recall is not warranted other regulatory response actions may be required to address investigative or unsatisfactory sample results.

8.2 Samples judged Satisfactory (S)

The establishment should be notified, however, no further action is required. External LSTS Reports of Analysis (ROA) may be provided to regulated party if requested.

8.3 Samples Judged as No Decision (ND)

For samples judged as no decision, discuss the results with the Supervisor / Regional Program Officer or designate who will review as necessary.

A. андерсэмплинг с использованием tomek links:

Один из предоставляемых методов называется «Tomek Links». «Links» в данном случае — пары элементов из разных классов, находящиеся поблизости.

Используя алгоритм мы в конечном итоге удалим элемент пары из большего набора, что позволит классификатору отработать лучше.

Источник

Additives and components analysis

Additives are chemicals added to the product during processing in order to preserve it in some manner, modify the colour, modify the taste, or alter the characteristics of the product. The application methods for these substances may vary which affects the distribution of the substance in the product.

Components analyses are used to assess and express the nutritional value of a food. To determine compositional parameters in a food product, the amount (concentration, percentage) of ingredients, components and nutrients such as fat, protein, sodium, sugars, moisture, etc. are analysed

B. оверсэмплинг со smote:

В SMOTE (Способ Передискретизации Синтезированных Меньшинств) мы создаём элементы в непосредственной близости от уже существующих в меньшем наборе.

Источник

from imblearn.over_sampling import SMOTE

smote = SMOTE(ratio='minority')
X_sm, y_sm = smote.fit_sample(X, y)

Но в imblearn существуют и другие способы андерсэмплинга (Cluster Centroids, NearMiss, и т.д.) и оверсэмплинга (ADASYN и bSMOTE), которые тоже могут пригодиться.

Chemical contaminants

Chemical contaminants are substances which are present in the food products as a result of the environmental conditions to which the food was exposed (for example allergens, sanitizer etc.). Organic contaminants concentrate in the lipid portion of the food whereas inorganic contaminants are more uniformly distributed throughout the muscle (protein) tissue or aquous portion of food (for example lead and paraformaldehyde, mercury, etc.)

Chemical indicators

Chemical indicators are substances which are produced from decomposition processes that are occurring in the food. Chemical testing is often used to corroborate results from sensory analysis. Quality indices include histamine, indole, and total volatile base nitrogen (TVBN), patuline in apple juice, peroxide value, etc.

Documentation

Complete the sample submission form and include a signed copy of the form in the shipping box.

The Chain of Custody form (CFIA/ACIA 5428 internal access only) must be completed and should be attached to the outside of the shipping box (to facilitate the signing of the form upon sample transfer).

Refer to the Enforcement and Investigation Services (EIS) document: Evidentiary Consideration for Samples for further information.

Food sampling flashcards | quizlet

Laboratory contact information

Consult the appropriate laboratory contact prior to taking legal samples, to ensure that the laboratory is able to handle the samples.

Obtaining the sample

Refer to FDR Subsection A.01.050 for complete details on sampling requirements. When taking a sample of an article pursuant to paragraph 23(1)(a) of the FDA, an inspector must inform the owner thereof or the person from whom the sample is being obtained of the inspector’s intention to submit the sample or a part thereof to an analyst for analysis or examination, and

1. Where, in the opinion of the inspector, division of the product to be sampled would not interfere with analysis or examination, obtain a sufficient quantity of product for a triplicate sample and
   1. divide the quantity into three parts,
   2. identify the three parts as the owner’s portion, the sample, and the duplicate sample and where only one part bears the label, that part shall be identified as the sample,
   3. seal each part in such a manner that it cannot be opened without breaking the seal, and
   4. deliver the part identified as the owner’s portion to the owner or the person from whom the sample was obtained and forward the sample and the duplicate sample to an analyst for analysis or examination.
2. Where, in the opinion of the inspector, division of the procured quantity would interfere with analysis or examination
   1. identify the entire quantity as the sample,
   2. seal the sample in such a manner that it cannot be opened without breaking the seal, and
   3. forward the sample to an analyst for analysis or examination.

Always choose sub-samples randomly from the same lot. At no time should a company representative choose the samples.

Product safety parameters and drug residue

Product safety parameters are those parameters which are used to curtail bacterial growth in a product and prolong the product shelf life. The parameters may be used in combination in a product or only one parameter may be controlled to prevent bacterial growth. Salt, water activity, and pH are included in this category.

Drug residue is residue that has resulted from the application of antibiotics or similar substances to animals (cattle, fish, bees, etc.) to prevent or treat disease. Tetracyclines, sulfonamides, and chloramphenicol are included in this category.

Sample identification

Write the sample number on all containers/packages of samples submitted to the laboratory. Avoid obscuring portions of the labels that are significant, such as the list of ingredients, label claim statements, product and brand name, lot code, etc.

Sample number and size

Consult with the Inspection Supervisor prior to taking legal samples. Generally, for extraneous material testing, select 24 individual containers with a minimum of 250 gram (g) or mL (milliliter) per container. For all other analyses, select 5 sub-samples with a minimum of 250 g or mL per sub-sample.

Security

• Take all the necessary precautions to ensure that the sample is kept in a controlled environment under lock and key from the time the sample is taken and the time it is shipped to the laboratory
• Seal the sample with yellow CFIA Official Seal tape in a manner that it cannot be opened without breaking the seal
• Clearly write “Legal Samples” on the box to ensure that receiving laboratory staff take proper precautions to protect the samples and ensure validity in court

Shipping

• Use a courier service that guarantees next-day delivery
• Do not ship legal samples with planned or ad hoc samples
• Pack shipments in such a way as to minimize the potential for damage occurring in transit

Surface contact sampling: replicate organism direct agar contact (rodac) plate method (for flat impervious surfaces)

1. Wash and dry hands
2. While wearing sterile gloves, open the bag containing the RODAC plates and only remove the plate that will be used.
3. Remove the plastic cover and carefully press the agar surface to the surface being sampled.
4. Apply uniform pressure on the back of the plate so that the entire agar surface contacts the area being sampled.
5. Remove the RODAC plate from the surface being sampled and replace the cover onto the RODAC plate.
6. Repeat the previous steps until all environmental surfaces are sampled using a new RODAC plate each time
7. Immediately label each RODAC plate with adequate information so that the plate can be matched to the sampling submission form. The markings must be legible and permanent.
8. Stack the plates (like 5 – 10) on top of each other and secure them with masking tape
9. Put the piled RODAC plates in sample bags or other suitable containers
10. Seal bag tightly
11. The sample should be well marked, for example site location, conditions, etc.
12. Place the sample into your carry bag/ice packed cooler
13. Wipe off the surface that was sampled with a sterile wet cloth or sanitizer (such as 70% ethanol)

Surface contact sampling: swab method

This method is used to sample any surface up to 1 meter squared (m2). When collecting environmental samples from inside large equipment (for example, milk drier), dress entirely in sterile clothing (such as head cover, facial hair cover, disposable overalls and overshoes). Ideally, a 900 cm2 (30 × 30 cm or equivalent) surface should be swabbed whenever possible.

Андерсэмплинг и оверсэмплинг с использованием imbalanced-learn

imbalanced-learn (imblearn) — это питоновская библиотека для борьбы с проблемами несбалансированных наборов данных.

Она содержит несколько различных методов для проведения ресэмплинга.

Простой случайный сэмплинг

Допустим, если вы хотите сделать выборку, где каждый элемент имеет равную вероятность быть выбранным.

Ниже мы выбираем 100 таких элементов из датасета.

sample_df = df.sample(100)

Резервуарный сэмплинг

Мне нравится такая формулировка задачи:

Допустим, у вас есть поток элементов большого неизвестного размера, по которым можно итерироваться только один раз.

Создайте алгоритм, произвольно выбирающий элемент из потока так, как если бы любой элемент мог быть выбран с равной вероятностью.

Как это сделать?

Допустим, нам надо выбрать 5 объектов из бесконечного потока, так чтобы каждый элемент в потоке мог быть выбран равновероятно.

import random
def generator(max):
    number = 1
    while number < max:
        number  = 1
        yield number
# Создаём генератор потока
stream = generator(10000)
# Делаем резервуарный сэмплинг
k=5
reservoir = []
for i, element in enumerate(stream):
    if i 1<= k:
        reservoir.append(element)
    else:
        probability = k/(i 1)
        if random.random() < probability:
            # Сохраняем элемент из потока, удаляя ранее добавленный 
             reservoir[random.choice(range(0,k))] = element
print(reservoir)
------------------------------------
[1369, 4108, 9986, 828, 5589]

Доказать, что каждый элемент мог быть выбран равновероятно можно математически.

Как?

Когда дело доходит до математики, лучше попытаться начать решение с небольшого частного случая.

Так что давайте рассмотрим поток, состоящий из 3-х элементов, где нам нужно выбрать только 2.

Мы видим первый элемент, сохраняем его в списке, так как в резервуаре ещё есть место. Мы видим второй элемент, сохраняем его в списке, так как в резервуаре ещё есть место.

Мы видим третий элемент. Здесь становится интереснее. Мы сохраним третий элемент с вероятностью 2/3.

Давайте теперь посмотрим итоговую вероятность первого элемента быть сохранённым:

Вероятность вытеснения первого элемента из резервуара равна вероятности третьего элемента быть выбранным, умноженной на вероятность что именно первый элемент из двух будет выбран для вытеснения. То есть:

2/3 * 1/2 = 1/3 

То есть конечная вероятность первого элемента быть сохранённым:

1 — 1/3 = 2/3 

Абсолютно такую же логику можно применить и для второго элемента, распространив её в дальнейшем на большее количество элементов при увеличении резервуара.

То есть каждый элемент будет сохранён с вероятностью 2/3 или в общем случае k/n.

Случайный андерсэмплинг и оверсэмплинг

Источник

Слишком часто в жизни встречаются несбалансированные наборы данных.

Широко применяемый в таком случае способ называется ресэмплинг (в русском переводе иногда говорят «передискретизация» — прим. перев.). Его суть заключается либо в удалении элементов из слишком большого набора (андерсэмплинг) и/или добавлении большего количества элементов в недостаточно большой набор (оверсэмплинг).

Давайте для начала создадим какие-нибудь несбалансированные наборы.

Стратифицированный сэмплинг

Допустим, нам нужно оценить среднее количество голосов за каждого кандидата на выборах. Голосование проходит в трёх городах:

В городе A живёт 1 миллион рабочих

В городе B живёт 2 миллиона художников

В городе C живёт 3 миллиона пенсионеров

Если мы попытаемся взять равновероятные выборки по 60 человек среди всего населения, то они наверняка будут разбалансированы относительно разных городов, а потому предвзяты, что приведёт к серьёзной ошибке в предсказаниях.

Если же мы специально сделаем выборку из 10, 20 и 30 человек из городов A, B и C соответственно, то ошибка будет минимальной.

На Python это можно сделать так:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    stratify=y, 
                                                    test_size=0.25)

Appendix 3 – sampling for microbiological analysis

All samples must accurately reflect microbiological conditions at the time that sampling is performed. To maintain sample integrity, follow the procedures listed below.

0 Appendices

For general inquiries related to this Operational Guidance Document, please follow established communication channels, including submitting an electronic Request for Action Form (e-RAF) – (internal access only).

0 Definitions

Unless specified below, definitions are located in either the:

Заключение

Алгоритмы — кровь науки о данных.

Сэмплинг — одна из важнейших областей в работе с данными и выше приведён только поверхностный обзор.

Хорошо выбранная стратегия сэмплинга может потянуть весь проект за собой. Выбранная плохо приведёт к ошибочным результатам. Поэтому выбор нужно делать с умом.

Appendix 2 – sampling for chemical analysis

Sample units chosen for chemical analysis should not undergo any adulteration (such as rinsing with water) which may change the chemistry results.

Please refer to General Sampling Instructions – Sampling Foods for Chemical Surveillance Programs.

Appendix 7 – random sampling procedure

A true random selection of pallets and the selection of individual packages from each case can be achieved by using a serial number system and random number tables or a device such as a statistics calculator or computer with random number generation capabilities.

The random selection of cases from the pallets can be conducted by using a lot numbering system and a random number generator.

Appendix 4 – sampling for extraneous material

Extraneous material can include many different types of materials. Materials that can be harmful because of their hardness, sharpness, size or shape are of particular concern. Examples of harmful extraneous materials include glass, wood-chips, insulation, metal, or sharp plastic greater than 2 mm in size.

To help determine if the extraneous material is hazardous, background information should be gathered on the type of material used in the facility, facility maintenance history, consumer complaints and evidence of possible tampering.

When consumer complaints or inspection evidence indicates contamination by potentially harmful extraneous material, the product may need to be detained and directed samples taken. Laboratory consultation is also recommended. See document Guidelines for the General Cleanliness of Food – Extraneous Material Overview for more information on risk assessment and acceptance sampling plan parameters.

Appendix 1 – legal sampling procedure

An inspector may perform legal sampling when it is anticipated that legal action is anticipated. Legal sampling demands more complex procedures for sample collection, submission and laboratory testing compared to routine product sampling.

In order to ensure that testing results are admissible in court, legal sampling procedures, as directed in FDR subsection A.01.050, must be strictly followed. Inspectors should consult with the Inspection Supervisor to determine if it is necessary to seek legal advice and consult with CFIA Enforcement and Investigation Services (EIS) prior to initiating legal sampling.

8 Results – analysis and interpretation

The LSTS will generate one of four possible sample test result outcomes: Satisfactory (S), Unsatisfactory (U), Investigative (I) and No Decision (ND). It is important to remember that the sample assessment is not directly related to compliance.

Appendix 5 – sampling for radiation analysis

Irradiation is regulated under FDR Division 26. Industry may make submissions to Health Canada to allow new uses of food irradiation. Health Canada will permit new uses of food irradiation only after a safety assessment, and only listed items may be irradiated.

The current list of foods that are permitted to be irradiated is found on the CFIA external website: Irradiated Foods.

Food may be exposed to ionizing radiation for a variety of purposes, including:

• To improve the safety of food by reducing levels of pathogens associated with food-borne disease such as E. coli and Salmonella
• Reduce microbiological growth causing spoilage and, thereby, extend shelf-life
• Reduce insect infestation
• Delay ripening of fruit and vegetables