Generative AI Models: GANs, VAEs, Transformers, and Diffusion

Reading Objectives

This page contains a English translation. We will compare GANs, VAEs, Transformers, and diffusion models, selecting the appropriate technology and implementation approach for business tasks, evaluating cost, security, resources, and quality, and gradually advancing the experiments into production use. Video script texts for four corresponding videos are retained after the main content.

One: How do the four generative models work?

GAN: Let generation and discrimination compete with each other

A Generative Adversarial Network (GAN) consists of two networks. The generator creates content, while the discriminator determines whether a given sample is real or generated. During training, the generator strives to produce more realistic outputs, and the discriminator works to improve its ability to distinguish between real and generated samples, with both networks adjusting through this competitive process.

GANs are commonly used in image generation, style transfer, and image-to-image translation. After training, the generation speed can be very fast, but the training process may be unstable and may suffer from "mode collapse": the generator repeatedly produces a limited set of similar results, failing to capture the diversity of the data.

For example, fashion designers can explore new designs by building upon existing styles, experimenting with variations while maintaining the brand's visual language. The original source notes that GANs are important in visual creation, but many newer image generation tasks also consider diffusion methods.

VAE: compressed representation, reconstruction from it, and variation

A Variational Autoencoder (VAE) uses an encoder to map inputs to a latent space with a probabilistic structure, and then a decoder reconstructs the data from this representation. Training simultaneously considers both reconstruction quality and the organization of the latent space.

This structure enables smooth transitions between different representations and facilitates exploration of property adjustments. As an example from the original source, product designers can attempt to make the appearance more minimalist or more rugged, while retaining certain functional features.

VAE training is typically more stable than GAN and may have lower computational requirements for certain tasks, but images generated independently can tend to be blurry. The actual performance still depends on the model, data, and specific task, and cannot be determined solely by category.

Transformer: Use attention to process context

The Transformer computes relationships between data elements through attention mechanisms. Inputs are represented as tokens, such as text fragments or image blocks; self-attention enables the model to assign different weights to various elements, capturing relationships between distant elements.

It is applicable to text generation, language processing, code, and document work, and can be extended to other data types. Using a contract draft as an example: adjust the text based on region, parties involved, and clause requirements; in human resources scenarios, it can draft personalized onboarding plans.

Note: Generating tokens one after another in sequence is a common approach used by many autoregressive language models, and not all Transformers operate this way. Legal documents such as contracts still require review by qualified professionals, and model architecture cannot guarantee correct legal applicability.

Diffusion models: learning to recover structure from noise

The diffusion method first considers a forward process: gradually adding noise to the data until its original structure is progressively destroyed; then learns the corresponding denoising process. During generation, it starts from noise and gradually recovers structured content through multiple steps.

It is widely used for text-to-image generation and visual concept exploration, allowing results to be guided by textual and other conditions. Marketing teams can develop visual campaign concepts based on this, while architectural designers can explore spatial layout ideas.

The original material describes diffusion models as having high image quality and strong control capabilities, while also requiring significant computational resources and being relatively slow in generation. This is a general comparison and should not be interpreted as a fixed ranking for all current models; new architectures, sampling methods, and deployment conditions can alter the results.

View textual explanation of the diagram

GAN trains generators and discriminators in an iterative manner; VAE encodes data into a latent distribution and then decodes it; Transformers use attention mechanisms to compute relationships between representations; diffusion models handle the noising process, gradually recovering structure from noise during generation. They do not belong to completely mutually exclusive classification hierarchies.

The diagrams illustrate competitive scenarios, compression and reconstruction, attention relationships, and step-by-step denoising; they are mechanistic illustrations, not product quality rankings.

comparison table

Note: These categories are not mutually exclusive product tags. A real system might simultaneously employ Transformer, VAE, and diffusion processes; model quality, speed, and hardware requirements are also influenced by scale, implementation, and task.

How to connect capabilities to the organization

The original list includes interface ecosystems such as GPT, Claude, Gemini, and services from Azure and AWS; visualization or workflow tools include Bubble, Webflow, Zapier, and Make. Hugging Face Spaces provides an environment for app showcasing and deployment, but it should not be considered a fully no-code platform. These names are for conceptual awareness, not current pricing, features, or procurement recommendations.

III. Costs, Safety, and Computing Resources

AI services may be charged per token, per API call, or per unit of computation. A token is a fragment processed by the model, and does not necessarily equal one word; tokenization methods vary by language and content.

The original material provides an assumption: $0.01 per 1,000 tokens, with monthly generation of 1,000 articles, each consisting of 500 tokens, totaling $5 for these outputs; if the volume increases to 100,000 articles, the cost becomes $500. This illustrates a linear scaling example and is not the current pricing for any service, and does not include input, retries, review, storage, or other fees.

Security and privacy must at least ask: where is the data being transmitted, how long is it retained, who has access, and whether the implementation meets applicable requirements. Sensitive data may require stricter access controls, contractual agreements, private environments, or on-premises deployment, merely ensuring functionality is available is insufficient.

Computing resources must also be matched to the task. The original material compares edge devices with lightweight models, regular servers with smaller models, and GPU clusters with large models; if using cloud-based APIs, the burden of maintaining model infrastructure in-house can be reduced, though continuous usage and data processing requirements must still be accounted for. Whether hardware is sufficient ultimately depends on actual testing of specific model size, precision, concurrency, and latency.

Four. First define quality, then make a choice

Realistic image generation can be evaluated using diffusion or GAN models; coherent text generation can be assessed with language models; systematic attribute manipulation can be studied using VAEs; music and audio typically require evaluation against models specialized in those domains. This correspondence provides a starting point and does not replace actual testing.

The evaluation output is divided into three levels.

• Technical quality : Is the final output realistic or natural? Is the content coherent? Can quality be maintained across multiple generations?
• Business adaptation Does it align with the organization's tone and brand? Has it accomplished the intended task? Do relevant users accept it?
• Preparation for implementation How much manual post-processing is required? Can the output format be integrated into existing systems? Will it still run efficiently with increased volume?

The most popular and largest model doesn't necessarily mean it's the best fit for your task. You should compare actual results using the same inputs and evaluation criteria.

Five stages from pilot to go-live

These cycles are for teaching reference only and do not constitute commitments to project timelines. When determining the next step, ask sequentially: Is the application clearly understood? Is the technical approach finalized? Is a prototype available and functional? Is testing sufficiently comprehensive? Address the missing step first, then proceed to deployment and ongoing monitoring.

During implementation, adopt a process combining AI-generated content with human editing, establishing a feedback loop, task-specific evaluation criteria, and prompting libraries. Initially focus on enhancing human capabilities, gradually accumulating experience.

Six. Map generative capabilities to business functions

The value and complexity here represent a screening rationale, not actual measured return on investment. Specific industry compliance, review, and integration costs may alter the judgment.

Value vs. Complexity Matrix

The original case, anonymized, describes an B2B software company that reduced document creation time by 60% after manual review. No verifiable source was provided, so this figure cannot be considered a reliable expectation for our own project.

Seven. Pre-implementation Check List

An organization needs clearly defined applications and measurable objectives, a project sponsor, technical capabilities aligned with the chosen path, specialized knowledge to evaluate content, a plan for process changes, appropriate ethical boundaries for usage, and metrics to measure success.

Note: Fine-tuning can be performed by managed services, and custom applications do not necessarily need to train large models on their own. Therefore, owning GPUs or having access to high-performance clusters is not a prerequisite for all approaches. Model training, application development, and model hosting should be accounted for separately.

Start with clearly defining the problem and conducting small pilot projects, establish evaluation criteria, and then gradually expand. Proceed to completion. Generative AI Suitability Exercise .

Video script one: Four generation methods

Generative AI is not exactly the same, especially in image tasks. The approach chosen will affect what can be generated, how fast it can be done, and how much resources are required. The original explanation compares four different architectures.

The first type is generative adversarial networks. Two networks engage in a creative competition: one generates content, and the other evaluates whether the content resembles real samples, continuously pushing each other to improve. It can quickly generate realistic images and is commonly used in image generation and style transfer. For example, fashion designers can maintain the brand's visual identity while exploring new variations.

The limitation is that the network may get stuck on similar outputs, lacking variation; the training process may also be unstable.

The second one is the variational autoencoder. It compresses input into an organized representation, then learns to reconstruct and generate controlled variations, which can be analogized to learning how to create different variations of a recipe.

Potential space can support smooth transitions between attributes. Designers can adjust certain features, such as simplicity or ruggedness, while attempting to preserve functional direction. Compared to GANs, its training is often more stable, but individual generated results may appear blurry.

The third is Transformer. The attention mechanism helps manage relationships within data, maintaining coherence over long contexts. It can be scaled to very large sizes and is widely used in text, language, and code.

The original explanation uses a draft contract as an example, adjusting the text according to regional and clause-specific requirements; human teams can also prepare onboarding plans for different employees. These are solely for drafting purposes and cannot replace professional review. One of the costs associated with large-scale implementation is higher computational demand.

The fourth type is diffusion models. First, consider gradually adding noise, then learn the reverse process. During generation, start from noise and progressively form structured content. Text conditions can guide image creation, enabling the marketing team to quickly explore visual concepts for activities without having to complete full-scale photography first.

The original explanation described it as having high image quality and good controllability, but with relatively high computation cost and slower speed. This was a summary at the time, not a fixed ranking for all models.

Selection should be based on requirements: rapid visual inspection, controlled attributes, coherent text, or detailed conditional imagery, with different evaluation criteria accordingly. Do not overlook output requirements simply because a particular architecture is popular; real systems may combine multiple approaches.

Video script two: Four implementation pathways

Using generative AI in work or business does not require a PhD in machine learning. Organizations can choose among four implementation approaches, depending on their technical resources and strategic objectives.

First, API integration: connect existing systems to cloud models. Applications send requests and receive results, without needing to maintain the full model infrastructure in-house. Suitable for organizations with basic technical resources who wish to integrate capabilities into their core workflows.

For example, a customer service team can integrate a language model to analyze tickets, assign them to the appropriate department, and flag complex issues for human review. This is a relatively quick way to start testing, but still requires handling permissions, errors, and costs.

Second, no-code platforms allow app creation through visual interfaces and pre-built components, reducing the need for line-by-line coding. Teams lacking technical expertise but requiring customizations can use them to build prototypes. For example, a marketing team can create a content tool linked to brand tone and approval workflows.

Third, off-the-shelf products come pre-configured for specific business functions, often requiring only minimal setup to get started. This approach is suitable when standard processes are in place and rapid implementation is desired; however, flexibility is typically limited. The original example uses resume analysis and sorting to illustrate scenarios involving human opportunities, which require careful examination of fairness and applicability conditions.

Fourth, custom development involves building systems tailored to specific needs, offering higher flexibility but also requiring strong technical expertise and maintenance capabilities. It may be considered if unique strategic value or highly specialized tasks are involved. For example, integrating multiple models with proprietary trading mechanisms to create a risk management system is feasible, though it entails greater investment.

Suggest starting with a simpler approach to demonstrate value, build a prototype and gain internal experience, before deciding whether a more complex implementation is needed. The goal is to deliver verifiable business value while building capability, rather than immediately choosing the most complex technology.

Video transcript three: How to make technical choices

Choosing a generation tool is not merely a matter of preference; it can impact organizational efficiency, costs, and rework. A wrong choice may lead to long-term waste, so it's essential to align business objectives, team capabilities, and budget considerations.

Many services charge by usage. The original explanation uses hypothetical monthly prices of about $5 for 1,000 articles and about $500 for 100,000 articles to show how increasing volume affects costs. A token does not always equal a word, and these are not actual current prices. A calculation must also include input, intermediate calls, retries and other costs.

First, look at the output quality requirements. For realistic visual effects, suitable image generation methods should be selected; for coherent long-form text, appropriate language models should be evaluated; for systematic attribute changes, the corresponding representation and control mechanisms should be studied.

Then look at purpose-specific adaptation. Editing text, images, realistic photos, and creative concepts may require different optimal approaches; one architectural name does not guarantee the best result.

Third is resource limitations. Cloud-based APIs can reduce the need for in-house infrastructure, but still incur ongoing usage costs; edge and mobile devices require models tailored to hardware specifications; large models or complex image tasks may demand more computational power. Real-world conditions should be tested, rather than judged solely by labels such as "standard server" or "GPU cluster."

Also check security and privacy: where are the data processed, how long does the service provider retain them, who has access, and does it comply with applicable requirements? Sensitive applications may require a private environment or more stringent controls, which also come with increased costs and complexity.

Evaluate the results from three aspects: technical quality: whether it is natural, coherent, and consistently stable; business alignment: whether it conforms to organizational style and achieves its intended purpose; implementation: how much manual editing is required and whether it can integrate with existing systems.

Make large-scale commitments only after first conducting small-scale pilot tests. The most advanced and complex models are not necessarily the best fit for the current task; instead, select a solution that aligns with the specific use case, budget, and technical capabilities.

Video script four: From pilot to production

Many projects face problems not just with whether the model works, but also with missing critical planning steps. Below is a five-stage explanation of what needs to be done, from idea to actual implementation.

Phase one defines purpose and measurable criteria. What specific problem is being addressed? How will success be measured? What quality must the output achieve? Clarification may take several days to weeks, potentially reducing months of rework later; exact duration still varies by project.

In the second stage, select the technology based on requirements: the model, implementation method, platform, costs, and resources must align. It's not acceptable to first purchase tools and then try to reshape the business to fit what the tools can handle.

Third-stage design and configuration: create interface sketches, connect services, establish reusable prompts, define workflows, and integrate with existing systems.

Stage 4: Testing and Improvement. Use diverse inputs to identify edge cases and quality issues, refine prompts based on results, provide feedback, and implement error handling. Thorough testing can reduce production issues and help determine whether output quality is maintained.

Stage five begins with deployment to a small number of users or limited use cases, with close monitoring of performance and costs. Unexpected outcomes may arise from usage-based pricing, so recording and controls are essential. The system gathers feedback and gradually scales up based on proven results, rather than expanding based on assumptions.

Staged progression can reduce risks and gradually build internal capabilities. Knowledge and experience gained from each pilot can support the next step.

Prioritize using a value-versus-complexity matrix: high-value, low-complexity content, emails, or basic Q&A should be addressed first for validation; more complex design or analysis can be treated as strategic projects; high-complexity innovation requires substantial experience and clear justification.

More effective strategies typically combine AI-generated content with human editing, expertise, and judgment. First demonstrate some value, then systematically build up capabilities, and finally extend into more strategically significant applications.

Related Reading

use Tool applicability practice Match the model with the task; when creating actual content, you can continue to refer. Content creation strategy .

Reference and Extended Reading

The following is an editorial addition.

• TensorFlow: Generative Adversarial Networks Tutorial : Chinese, view generator, discriminator, and training process.
• Hugging Face: Getting Started with Diffusion Models : English, understand the denoising process and conditional generation.
• Google: Overview of Transformers : Chinese, query attention and sequence processing.